Moving object control apparatus, moving object control method, and moving object

ABSTRACT

The present technology relates to a moving object control apparatus, a moving object control method, and a moving object that make it possible to properly perform driving of a moving object in accordance with a person getting on the moving object or a candidate therefor and a person getting off the moving object or a candidate therefor. The moving object control apparatus includes a driving control unit that performs, on the basis of recognition of at least one of a candidate for getting on the moving object or a candidate for getting off the moving object, acceleration/deceleration control of the moving object. The present technology is applicable to, for example, a moving object such as a bus and a taxi, or a control apparatus that controls automatic driving or semi-automatic driving of the moving object.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. § 371 as a U.S.National Stage Entry of International Application No. PCT/JP2017/029253,filed in the Japanese Patent Office as a Receiving Office on Aug. 14,2017, which claims priority to Japanese Patent Application NumberJP2016-165709, filed in the Japanese Patent Office on Aug. 26, 2016,each of which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present technology relates to a moving object control apparatus, amoving object control method, and a moving object, and particularly to amoving object control apparatus, a moving object control method, and amoving object that are suitable for performing automatic driving orsemi-automatic driving.

BACKGROUND ART

In the past, a system for performing driving assistant of a vehiclecapable of platooning by automatic driving (see, for example, PatentLiterature 1).

CITATION LIST Patent Literature

-   Patent Literature 1: Japanese Patent Application Laid-open No.    2000-285363

DISCLOSURE OF INVENTION Technical Problem

However, the invention described in Patent Literature 1 is directed tothe vehicle capable of platooning, and it is necessary to developinfrastructure such as a traffic control system, a dedicated road, and aboarding platform having a push button device. Therefore, a case where amoving object including a vehicle such as a bus and a taxi performsautomatic driving or semi-automatic driving by itself is not considered.

The present technology has been made in view of the above-mentionedcircumstances to make it possible to properly perform driving of amoving object on the basis of a person getting on the moving object or acandidate therefor and a person getting off the moving object or acandidate therefor.

Solution to Problem

A moving object control apparatus according to a first aspect of thepresent technology includes a driving control unit that performs, on abasis of recognition of at least one of a candidate for getting on amoving object or a candidate for getting off the moving object,acceleration/deceleration control of the moving object.

The driving control unit may perform, on a basis of at least one of therecognition of the candidate for getting on or the recognition of thecandidate for getting off, stop control of the moving object.

The driving control unit may perform, on a basis of at least one ofrecognition of getting on the moving object of the candidate for gettingon or recognition of getting off the moving object of the candidate forgetting off, starting control of the moving object.

The recognition of the candidate for getting on may be performed on anarea in a vicinity of a predetermined stop candidate point of the movingobject.

A priority recognition area may be set on a basis of a predeterminedreference position in the stop candidate point, the candidate forgetting on being preferentially recognized in the priority recognitionarea.

The reference position may be at least one of a sign indicating the stopcandidate point, a predetermined structure at the stop candidate point,or a predetermined boarding position at the stop candidate point.

The priority recognition area may be further set on a basis of at leastone of a configuration of the stop candidate point or a situation aroundthe stop candidate point.

The reference position may be changed depending on a situation aroundthe stop candidate point.

The area to be recognized may be set to a predetermined area in apredetermined direction with respect to the moving object.

The recognition of the candidate for getting on may be performed on abasis of at least one of a face, an orientation of a line of sight, aposition, or behavior of a person recognized in an image captured by animaging unit of the moving object.

The recognition of the candidate for getting on may be further performedon a basis of an image captured by an imaging apparatus outside themoving object.

The recognition of the candidate for getting on or the candidate forgetting off may be performed on a basis of face authentication using animage of a face of the candidate for getting on or the candidate forgetting off or data indicating a feature amount of the face, the imageand the data being acquired in advance.

The moving object control apparatus may further include a plurality ofimaging units for recognizing the candidate for getting on or thecandidate for getting off in different directions or different positionsin the moving object, in which the imaging unit used for processing ofrecognizing the candidate for getting on or the candidate for gettingoff may be changed between when the moving object moves and when themoving object stops.

The recognition of the candidate for getting off may be performed on abasis of at least one of a position or behavior of an occupant of themoving object in an image obtained by imaging an inside of the movingobject.

The moving object control apparatus may further include a user interfacecontrol unit that controls notification of a recognition result of thecandidate for getting on.

The user interface control unit may superimpose, on an image captured byan imaging unit of the moving object, information indicating therecognition result of the candidate for getting on, and displays theimage.

The user interface control unit may superimpose, on a field of view ofan occupant of the moving object, information indicating the recognitionresult of the candidate for getting on, for display.

A moving object control method according to a first aspect of thepresent technology includes a driving control step of performing, on abasis of recognition of at least one of a candidate for getting on amoving object or a candidate for getting off the moving object,acceleration/deceleration control of the moving object.

A moving object according to a second aspect of the present technologyincludes a driving control unit that performs, on a basis of recognitionof at least one of a candidate for getting on or a candidate for gettingoff, acceleration/deceleration control.

In the first aspect or the second aspect of the present technology,acceleration/deceleration control is performed on the basis ofrecognition of at least one of a candidate for getting on or a candidatefor getting off.

Advantageous Effects of Invention

In accordance with the present technology, it is possible to properlyperform driving of a moving object in accordance with a person gettingon the moving object or a candidate therefor or a person getting off themoving object or a candidate therefor.

It should be noted that the effect described here is not necessarilylimitative and may be any effect described in the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing a configuration example of an embodiment ofa vehicle to which the present technology is applied.

FIG. 2 is a diagram showing an example of the connection state ofrespective units of the vehicle.

FIG. 3 is a diagram for describing a method of generating a compositeimage.

FIG. 4 is a diagram showing a part of a configuration example of afunction of a vehicle control unit.

FIG. 5 is a diagram showing an arrangement example of cameras in a bus.

FIG. 6 is a flowchart for describing a first embodiment of automaticstop/departure processing.

FIG. 7 is a flowchart for describing the first embodiment of automaticstop/departure processing.

FIG. 8 is a diagram showing a type of a stop.

FIG. 9 is a diagram showing an example of a stop including a waitingroom.

FIG. 10 is a diagram showing an example of a stop including a pluralityof signs.

FIG. 11 is a diagram showing a first example of a priority recognitionarea.

FIG. 12 is a diagram showing a second example of the priorityrecognition area.

FIG. 13 is a diagram showing a third example of the priority recognitionarea.

FIG. 14 is a diagram showing a fourth example of the priorityrecognition area.

FIG. 15 is a diagram showing a fifth example of the priority recognitionarea.

FIG. 16 is a diagram for describing a first example of processing ofrecognizing a candidate for getting on.

FIG. 17 is a diagram for describing a second example of the processingof recognizing a candidate for getting on.

FIG. 18 is a diagram for describing an example of a method of switchingthe image used for the processing of recognizing a candidate for gettingon.

FIG. 19 is a flowchart for describing a second embodiment of theautomatic stop/departure processing.

FIG. 20 is a flowchart for describing a third embodiment of theautomatic stop/departure processing.

FIG. 21 is a diagram showing a first example of an assist screen.

FIG. 22 is a diagram showing a second example of the assist screen.

FIG. 23 is a diagram showing a third example of the assist screen.

FIG. 24 is a diagram showing a fourth example of the assist screen.

MODE(S) FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments for carrying out the present technology will bedescribed. Description will be made in the following order.

1. Basic Configuration Example of Vehicle

2. First embodiment (Example Applied to Bus that Performs AutomaticDriving)

3. Second embodiment (First Example Applied to Taxi that PerformsAutomatic Driving)

4. Third embodiment (Second Example Applied to Taxi that PerformsAutomatic Driving)

5. Fourth embodiment (Example Applied to User Interface of Vehicle thatPerforms Semi-Automatic Driving)

6. Modified example

7. Others

1. Basic Configuration Example of Vehicle

FIG. 1 is a diagram showing a basic configuration example of a vehicleto which the present technology is applied.

A vehicle 11 shown in FIG. 1 includes a front sensing camera 21, a frontcamera ECU (Electronic Control Unit) 22, a position informationacquisition unit 23, a display unit 24, a communication unit 25, asteering mechanism 26, a radar 27, a lidar 28, a side view camera 29, aside view camera ECU 30, an integrated ECU 31, a front view camera 32, afront view camera ECU 33, a braking device 34, an engine 35, a generator36, a driving motor 37, a battery 38, a rear view camera 39, a rear viewcamera ECU 40, a vehicle speed detection unit 41, an in-vehicle camera42 and an in-vehicle camera ECU 43.

The units provided in the vehicle 11 are connected to each other by abus for CAN (Controller Area Network) communication, another connectionline, and the like. However, in order to make the figure easy to see,the bus, the connection line, and the like are drawn withoutparticularly distinguishing them.

The front sensing camera 21 includes, for example, a camera dedicated tosensing disposed in the vicinity of a room mirror in the vehicle, imagesthe front of the vehicle 11 as a subject, and outputs the resultingsensing image to the front camera ECU 22.

The front camera ECU 22 appropriately performs processing of improvingthe image quality or the like on the sensing image supplied from thefront sensing camera 21, and then performs image recognition on thesensing image, thereby detecting an arbitrary object such as a whiteline and a pedestrian from the sensing image. The front camera ECU 22outputs the result of image recognition to the bus for CANcommunication.

The position information acquisition unit 23 includes, for example, aposition information measuring system such as a GPS (Global PositioningSystem), detects the position of the vehicle 11, and outputs theposition information indicating the detection result to the bus for CANcommunication.

The display unit 24 includes, for example, a liquid crystal displaypanel, and is disposed at a predetermined position in the vehicle suchas the center position of an instrument panel and the inside of a roommirror. Further, the display unit 24 may be a transmissive displaysuperimposed and provided on a windshield part, or a display of a carnavigation system. The display unit 24 displays various images under thecontrol of the integrated ECU 31.

The communication unit 25 transmits/receives information to/from aperipheral vehicle, a portable terminal device possessed by apedestrian, a roadside device, an external server, or the like byvarious kinds of wireless communication such as inter-vehiclecommunication, vehicle-to-pedestrian communication, and road-to-vehiclecommunication. For example, the communication unit 25 performsinter-vehicle communication with a peripheral vehicle, receives, fromthe peripheral vehicle, peripheral vehicle information includinginformation indicating the number of occupants and the travelling state,and supplies it to the integrated ECU 31.

The steering mechanism 26 performs control of the traveling direction ofthe vehicle 11, i.e., steering angle control, in accordance with thedriver's steering wheel operation or the control signal supplied fromthe integrated ECU 31. The radar 27 is a distance measuring sensor thatmeasures the distance to an object such as a vehicle and a pedestrian ineach direction such as forward and backward by using electromagneticwaves such as millimeter waves, and outputs the result of measuring thedistance to the object to the integrated ECU 31 or the like. The lidar28 is a distance measuring sensor that measures the distance to anobject such as a vehicle and a pedestrian in each direction such asforward and backward by using light waves, and outputs the result ofmeasuring the distance to the object to the integrated ECU 31 or thelike.

The side view camera 29 is, for example, a camera disposed in a casingof a side mirror or in the vicinity of the side mirror, captures animage of the side (hereinafter, referred to also as the side image) ofthe vehicle 11 including an area to be a blind spot of the driver, andsupplies it to the side view camera ECU 30.

The side view camera ECU 30 performs image processing of improving theimage quality such as white balance adjustment on the side imagesupplied from the side view camera 29, and supplies the obtained sideimage to the integrated ECU 31 via a cable different from the bus forCAN communication.

The integrated ECU 31 includes a plurality of ECUs such as a drivingcontrol ECU 51 and a battery ECU 52 arranged at the center of thevehicle 11, and controls the operation of the entire vehicle 11.

For example, the driving control ECU 51 is an ECU realizing an ADAS(Advanced Driving Assistant System) function or an automatic driving(Self driving) function, and controls the driving (travelling) of thevehicle 11 on the basis of various kinds of information such as theimage recognition result from the front camera ECU 22, the positioninformation from the position information acquisition unit 23, theperipheral vehicle information supplied from the communication unit 25,the measurement results from the radar 27 and the lidar 28, the resultof detecting the vehicle speed from the vehicle speed detection unit 41,and the like. That is, the driving control ECU 51 controls the steeringmechanism 26, the braking device 34, the engine 35, the driving motor37, and the like to control the driving of the vehicle 11.

Note that in the integrated ECU 31, a dedicated ECU may be provided foreach of the functions including the ADAS function, the automatic drivingfunction, and the like.

Further, the battery ECU 52 controls power supply or the like by thebattery 38.

The front view camera 32 includes, for example, a camera disposed in thevicinity of a front grille, captures an image of the front (hereinafter,referred to also as the front image) of the vehicle 11 including an areato be a blind spot of the driver, and supplies it to the front viewcamera ECU 33.

The front view camera ECU 33 performs image processing of improving theimage quality such as white balance adjustment on the front imagesupplied from the front view camera 32, and supplies the obtained frontimage to the integrated ECU 31 via a cable different from the bus forCAN communication.

The braking device 34 operates in accordance with the driver's brakingoperation or the control signal supplied from the integrated ECU 31, andstops or decelerates the vehicle 11. The engine 35 is a power source ofthe vehicle 11, and is driven in accordance with the control signalsupplied from the integrated ECU 31.

The generator 36 is controlled by the integrated ECU 31, and generatespower in accordance with driving of the engine 35. The driving motor 37is a power source of the vehicle 11, receives power supply from thegenerator 36 or the battery 38, and is driven in accordance with thecontrol signal supplied from the integrated ECU 31. Note that whether todrive the engine 35 or the driving motor 37 during travelling of thevehicle 11 is appropriately switched by the integrated ECU 31.

The battery 38 includes, for example, a battery of 12 V or a battery of200 V, and supplies power to the respective units of the vehicle 11 inaccordance with the control of the battery ECU 52.

The rear view camera 39 includes, for example, a camera disposed in thevicinity of a license plate of a tailgate, captures an image of the rearside (hereinafter, referred to also as the rear image) of the vehicle 11including an area to be a blind spot of the driver, and supplies it tothe rear view camera ECU 40. For example, the rear view camera 39 isactivated when a shift lever (not shown) is moved to the position of thereverse (R).

The rear view camera ECU 40 performs image processing of improving theimage quality such as white balance adjustment on the rear imagesupplied from the rear view camera 39, and supplies the obtained rearimage to the integrated ECU 31 via a cable different from the bus forCAN communication.

The vehicle speed detection unit 41 is a sensor that detects the vehiclespeed of the vehicle 11, and supplies the result of detecting thevehicle speed to the integrated ECU 31. Note that in the vehicle speeddetection unit 41, from the result of detecting the vehicle speed, theacceleration, or the differential of the acceleration may be calculated.For example, the calculated acceleration is used for estimating the timeuntil the collision of the vehicle 11 with an object.

The in-vehicle camera 42 includes, for example, a camera installedinside the vehicle 11, captures an image of the inside of the vehicle 11(hereinafter, referred to also as the in-vehicle image), and supplies itto the in-vehicle camera ECU 43.

The in-vehicle camera ECU 43 performs, on the in-vehicle image suppliedfrom the in-vehicle camera 42, image processing for improving the imagequality such as white balance adjustment, and supplies the obtainedin-vehicle image to the integrated ECU 31 via a cable different from thebust for CAN communication.

Note that hereinafter, the image obtained by capturing the outside ofthe vehicle 11, such as the sensing image, the side image, the frontimage, and the rear image, will be referred to also as thevehicle-exterior image.

Further, in the vehicle 11, as shown in FIG. 2, a plurality of unitsincluding a front camera module 71, the communication unit 25, thedriving control ECU 51, the steering mechanism 26, the braking device34, the engine 35, and the driving motor 37 are connected to each othervia a bus 72 for CAN communication. Note that the componentscorresponding to those in FIG. 1 are denoted by the same referencesymbols in FIG. 2, and description thereof will be appropriatelyomitted.

In this example, the front camera module 71 includes a lens 81, an imagesensor 82, the front camera ECU 22, and an MCU (Module Control Unit) 83.

Further, the lens 81 and the image sensor 82 constitute the frontsensing camera 21, and the image sensor 82 includes, for example, a CMOS(Complementary Metal Oxide Semiconductor) image sensor.

In the front camera module 71, light from a subject is collected on theimaging surface of the image sensor 82 by the lens 81. The image sensor82 captures a sensing image by photoelectrically converting light thathas entered from the lens 81, and supplies it to the front camera ECU22.

The front camera ECU 22 performs, for example, gain adjustment, whitebalance adjustment, HDR (High Dynamic Range) processing, and the like onthe sensing image supplied from the image sensor 82, and then performsimage recognition on the sensing image.

In the image recognition, for example, a white line, a curb stone, apedestrian, a vehicle, a headlight, a brake light, a road sign, a timeuntil a collision with the forward vehicle, and the like are recognized(detected). The recognition results of the image recognition areconverted into signals in a format for CAN communication by the MCU 83,and output to the bus 72.

Further, information supplied from the bus 72 is converted into a signalin a format defined for the front camera module 71 by the MCU 83, andsupplied to the front camera ECU 22.

The driving control ECU 51 appropriately controls the steering mechanism26, the braking device 34, the engine 35, the driving motor 37, aheadlight (not shown), and the like on the basis of the result of imagerecognition output from the MCU 83 to the bus 72 and informationsupplied from other units such as the radar 27 and the lidar 28.Accordingly, driving control such as change of the travelling direction,braking, acceleration, and starting, warning notification control,headlight switching control, and the like are realized.

Further, in the case where the driving control ECU 51 realizes theautomatic driving function or the like, for example, the locus of theposition of the target object may be further recognized by the drivingcontrol ECU 51 from the image recognition result at each time obtainedby the front camera ECU 22, and such a recognition result may betransmitted to an external server via the communication unit 25. In sucha case, for example, in the server, learning such as a deep neuralnetwork is performed, and a necessary dictionary or the like isgenerated and transmitted to the vehicle 11. In the vehicle 11, thedictionary or the like obtained in this way is received by thecommunication unit 25, and the received dictionary or the like is usedfor various predictions and the like in the driving control ECU 51.

Note that of the controls performed by the driving control ECU 51,control that can be realized from only the result of image recognitionon the sensing image may be performed not by the driving control ECU 51but by the front camera ECU 22.

Further, in the vehicle 11, by displaying a composite image on thedisplay unit 24 at the time of parking, for example, the around viewmonitor function is realized.

That is, as shown in FIG. 3, the front image, the rear image, and theside image obtained by the respective units are supplied, via a cabledifferent from the bus for CAN communication, to an image compositionECU 101 provided in the integrated ECU 31, and a composite image isgenerated from the images. Note that the components corresponding tothose in FIG. 1 are denoted by the same reference symbols in FIG. 3, anddescription thereof will be appropriately omitted.

In FIG. 3, as the side view camera 29 shown in FIG. 1, a side viewcamera 29L disposed on the left side of the vehicle 11 and a side viewcamera 29R disposed on the right side of the vehicle 11 are provided.Further, as the side view camera ECU 30, a side view camera ECU 30Ldisposed on the left side of the vehicle 11 and a side view camera ECU30R disposed on the right side of the vehicle 11 are provided.

To the image composition ECU 101, the front image obtained by the frontview camera 32 is supplied from the front view camera ECU 33 and therear image obtained by the rear view camera 39 is supplied from the rearview camera ECU 40. Further, to the image composition ECU 101, the sideimage obtained by the side view camera 29L (hereinafter, particularlyreferred to also as the left side image) is supplied from the side viewcamera ECU 30L and the side image obtained by the side view camera 29R(hereinafter, particularly referred to also as the right side image) issupplied from the side view camera ECU 30R.

The image composition ECU 101 generates, on the basis of the suppliedimages, a composite image in which the front image, the rear image, theleft side image, and the right side image are arranged in correspondingareas, and supplies the obtained composite image to the display unit 24for display. The driver is capable of safely and easily parking thevehicle 11 by driving the vehicle 11 while watching the composite imagedisplayed in this way. Note that the integrated ECU 31 may control thedriving of the vehicle 11 on the basis of the composite image, and parkthe vehicle 11.

Note that the side view camera ECU 30, the front view camera ECU 33, andthe rear view camera ECU 40 may detect an arbitrary object outside thevehicle, such as a white line and a pedestrian, from the side image, thefront image, or the rear image, similarly to the front camera ECU 22.Further, the in-vehicle camera ECU 43 may detect an arbitrary objectinside the vehicle, such as an occupant, from the in-vehicle image.

Further, the number or arrangement of cameras provided in the vehicle 11described above is an example, and appropriately changed depending onthe size, shape, of the like of the vehicle 11.

FIG. 4 shows a part of a configuration example of a function of avehicle control unit 151 of the vehicle 11. The vehicle control unit 151is realized by, for example, the front camera ECU 22, the side viewcamera ECU 30, the integrated ECU 31, the front view camera ECU 33, therear view camera ECU 40, the in-vehicle camera ECU 43, and the like. Thevehicle control unit 151 includes a monitoring unit 161, a situationrecognition unit 162, a driving control unit 163, a stop positionsetting unit 164, an imaging control unit 165, and an UI (UserInterface) control unit 166.

The monitoring unit 161 monitors the periphery and the inside of thevehicle 11. The monitoring unit 161 includes a vehicle-exteriormonitoring unit 171 and an in-vehicle monitoring unit 172.

The vehicle-exterior monitoring unit 171 monitors the outside of thevehicle 11 on the basis of the vehicle-exterior image, the positioninformation from the position information acquisition unit 23, varioustypes of information acquired from the outside via the communicationunit 25, the measurement result from the radar 27, the measurementresult from the lidar 28, the detection results by various sensors (notshown) provided in the vehicle 11, and the like. For example, thevehicle-exterior monitoring unit 171 detects an arbitrary object aroundthe vehicle 11, such as another vehicle, a human, and a white line.Further, for example, in the case where the vehicle 11 is a vehicle forcarrying passengers, such as a bus and a taxi, the vehicle-exteriormonitoring unit 171 performs processing of recognizing a candidate forgetting on predicted to get on the vehicle 11.

The in-vehicle monitoring unit 172 monitors, on the basis of thein-vehicle image, the detection results of various sensors (not shown)provided in the vehicle 11, and the like, the inside of the vehicle 11.For example, the in-vehicle monitoring unit 172 detects an arbitraryobject such as an occupant inside the vehicle. Further, for example, inthe case where the vehicle 11 is a vehicle for carrying passengers, suchas a bus and a taxi, the in-vehicle monitoring unit 172 performsprocessing of recognizing a candidate for getting off predicted to getoff the vehicle 11.

The situation recognition unit 162 recognizes the situation where thevehicle 11 is placed, on the basis of the position information from theposition information acquisition unit 23, the information acquired fromthe outside via the communication unit 25, the monitoring result fromthe monitoring unit 161, the detection results of various sensors (notshown) provided in the vehicle 11, and the like. Examples of thesituation recognized by the situation recognition unit 162 include thecurrent position, the current time, the weather, the temperature, thewind speed, the surrounding brightness, the road surface condition, andthe time.

The driving control unit 163 controls driving of the vehicle 11 on thebasis of the information acquired from the outside via the communicationunit 25, the detection result of the vehicle speed from the vehiclespeed detection unit 41, the monitoring result from the monitoring unit161, the situation recognition result from the situation recognitionunit 162, the driver's operation, and the like. For example, the drivingcontrol unit 163 performs starting control, stop control,acceleration/deceleration control, braking control, direction control,and the like of the vehicle 11.

Further, for example, the driving control unit 163 controls automaticdriving or semi-automatic driving of the vehicle 11. Here, the automaticdriving represents that all of the driving of the vehicle 11 isautomated and the vehicle 11 autonomously moves without the driver'soperation. The semi-automatic driving represents that a part of thedriving of the vehicle 11 is automated. The range of automation of thesemi-automatic driving varies from the one in which the driver performsalmost all of the driving and only a part of functions, such as brakingcontrol, is automated to assist the driver to the one in which thevehicle 11 autonomously performs almost all of the driving and thedriver performs a part of the operation in a supplementary manner, forexample. Further, for example, the driving control unit 163 performscontrol for the vehicle 11 to travel with a predetermined route orschedule, and control for reaching a predetermined destination.

Further, for example, the driving control unit 163 performs control ofopening/closing the door or window of the vehicle 11, control ofopening/closing the door lock, control of turning on the headlight,control of tilting of the vehicle body, power supply control, and thelike.

The stop position setting unit 164 sets the stop position of the vehicle11 on the basis of the monitoring result from the monitoring unit 161,the situation recognition result from the situation recognition unit162, and the like.

The imaging control unit 165 controls imaging by the front sensingcamera 21, the side view camera 29, the front view camera 32, the rearview camera 39, and the in-vehicle camera 42 on the basis of themonitoring result from the monitoring unit 161, the situationrecognition result from the situation recognition unit 162, the drivingcontrol information from the driving control unit 163, and the like.

The UI control unit 166 controls the display unit 24, a voice outputunit (not shown), and the like to control the user interface for theoccupant such as the driver, on the basis of the monitoring result fromthe monitoring unit 161, the situation recognition result from thesituation recognition unit 162, the driving control information from thedriving control unit 163, and the like.

2. First Embodiment

Next, a first embodiment of the present technology will be describedwith reference to FIG. 5 to FIG. 18. The first embodiment of the presenttechnology is an embodiment in the case where the vehicle 11 includes abus that performs automatic driving.

2-1. Arrangement Example of Cameras

FIG. 5 schematically shows an arrangement example of cameras in a bus201 as the vehicle 11 in FIG. 1.

On the bus 201, a camera 221F1 to a camera 221B are arranged as imagingunits. Any one of the camera 221F1 to the camera 221B may be a stereocamera including two or more cameras, and may be capable of measuringthe distance to an imaged object by the parallax of the two or morecameras. Further, any one of the camera 221F1 to the camera 221B mayinclude a wide-angle camera having a viewing angle of not less than 120degrees, particularly favorably, not less than 180 degrees.

The camera 221F1 and the camera 221F2 correspond to the front sensingcamera 21 or the front view camera 32 in FIG. 1. The camera 221F1 isdisposed, for example, in the vicinity of a room mirror (not shown) ofthe bus 201 and in the vicinity of the upper end in the up and downdirection at the center in the right and left direction of a windshield211, and used for mainly imaging the front of the bus 201. The camera221F2 is disposed, for example, above a dashboard (not shown) of the bus201 and in the vicinity of the lower end in the up and down direction atthe center in the right and left direction of the windshield 211, andused for mainly imaging the front of the bus 201. Note that the camera221F1 is used for imaging the front of the bus 201 which is more distantthan the position to be imaged by the camera 221F2, and the camera 221F2is used for imaging the front of the bus 201 which is nearer than theposition to be imaged by the camera 221F1.

The camera 221L1 to the camera 221L4 correspond to the side view camera29L in FIG. 3, and the camera 221R corresponds to the side view camera29R in FIG. 3. The camera 221L1 is disposed, for example, in thevicinity of a left side mirror 212L of the bus 201, and used for mainlyimaging the left side of the bus 201. The camera 221L2 is disposed, forexample, in the vicinity of an entrance door 213 of the bus 201, andused for mainly imaging the vicinity of the entrance door 213. Thecamera 221L3 is disposed, for example, in the vicinity of an exit door214 of the bus 201, and used for mainly imaging the vicinity of the exitdoor 214. The camera 221L4 is disposed, for example, in the vicinity ofthe rear end of the left side of the bus 201, and used for mainlyimaging the left side of the bus 201. Note that the camera 221L1 is usedfor imaging the left side of the bus 201 which is in front of the camera221L4, and the camera 221L4 is used for imaging the left side of the bus201 which is behind the camera 221L4. The camera 221R is disposed, forexample, in the vicinity of a right side mirror 212R of the bus 201, andused for mainly imaging the right side of the bus 201.

The camera 221B corresponds to the rear view camera 39 in FIG. 1. Thecamera 221B is disposed, for example, in the vicinity of the upper endin the up and down direction at the center in the right and leftdirection of the back surface of the bus 201, and use for mainly imagingthe rear of the bus 201.

The camera 221IN corresponds to the in-vehicle camera 42 in FIG. 1. Thecamera 221IN is disposed, for example, in the vicinity of the roommirror (not shown) of the bus 201, and used for mainly imaging theinside of the vehicle. Note that a plurality of cameras 221IN may beprovided in the vehicle so that no blind spot occurs inside the bus 201.

The images (vehicle-exterior images) captured by the camera 221F1, thecamera 221F2, the camera 221L1 to the camera 221L4, the camera 221R, andthe camera 221B are used for detecting an arbitrary object outside thevehicle, such as a white line and a pedestrian. The image (in-vehicleimage) captured by the camera 221IN is used for detecting an arbitraryobject such as an occupant inside the vehicle.

2-2. Automatic Stop/Departure Processing

Next, automatic stop/departure processing performed by the bus 201 willbe described with reference to the flowcharts of FIG. 6 and FIG. 7. Notethat this processing is started when an ACC (accessory) power source ofthe bus 201 is turned on, and finished when the ACC power source isturned off, for example.

In Step S1, the driving control unit 163 determines whether or not ithas approached a stop candidate point. The stop candidate pointrepresents, for example, a bus stop where the bus 201 is scheduled tostop next (hereinafter, referred to simply as the stop).

The vehicle control unit 151 detects the distance to the next stopfirst. As the method of detecting the distance to the next stop, anarbitrary method can be adopted.

For example, the situation recognition unit 162 detects the distance tothe next stop on the basis of the current position of the vehicle 11detected by the position information acquisition unit 23 and theposition of the next stop in the previously held map information.

Alternatively, for example, the vehicle-exterior monitoring unit 171detects a stop in the travelling direction of the bus 201 and furtherdetects the distance to the stop, on the basis of the vehicle-exteriorimage and the detection results of the radar 27 and the lidar 28.

Alternatively, for example, the situation recognition unit 162 detectsthe distance to the next stop on the basis of the strength of radiowaves that are emitted from the communication apparatus provided in thenext stop and then received by the communication unit 25.

Alternatively, for example, the situation recognition unit 162 detectsthe distance to the next stop on the basis of the information regardingthe position of the next stop or the distance to the next stop receivedby the communication unit 25 from the communication apparatus providedin the next stop.

In the case where the distance to the next stop is not less than apredetermined threshold value or detection of the distance to the nextstop has failed, the driving control unit 163 determines that it is notapproaching the stop candidate point. This determination processing isrepeatedly executed at a predetermined timing until it is determinedthat it has approached the stop candidate point. Then, in the case wherethe distance to the next stop is less than the threshold value, thedriving control unit 163 determines that it has approached the stopcandidate point, and the processing proceeds to Step S2.

In Step S2, the bus 201 enters a stop lane under the control of thedriving control unit 163. Here, the stop lane represents a lane wherethe bus 201 stops at the next stop. For example, in the case where adedicated bus lane is provided, the dedicated bus lane is the stop lane.Meanwhile, in the case where no dedicated bus lane is provided, the lanebefore the next stop, e.g., the left end lane is the stop lane in thecase of left-hand traffic.

Further, the bus 201 moves to the end on the side of the stop in thestop lane under the control of the driving control unit 163. Further,the UI control unit 166 controls a voice output apparatus or the like(not shown) to output an announcement for guiding the next stop.

In Step S3, the vehicle-exterior monitoring unit 171 sets a stopreference position.

Specifically, the vehicle-exterior monitoring unit 171 detects a stopfirst on the basis of the vehicle-exterior image.

FIG. 8 and FIG. 9 show types of the stop.

For example, there is a stop where only a sign indicating the name ofthe stop, the timetable, and the like is installed. A sign 301 to a sign303 in FIG. 8 are each an example of the sign of the stop. As shown inthis example, there are various shapes of the sign.

Further, for example, there is a stop where a bus shelter provided witha roof for protecting against wind, rain, sunshine and the like isinstalled. A bus shelter 304 in FIG. 8 is an example thereof. Some busshelters have a wall surface. Further, there is also a stop where a signas well as a bus shelter are installed. Alternatively, there is also astop where the name of the stop, the timetable, and the like areindicated in a bus shelter and no sign is installed.

Further, for example, there is a stop where a waiting room is installed.FIG. 9 shows an example of the stop where a waiting room is installed.In this example, a sign 311 is installed at the end of a sidewalk 313 onthe side of a roadway 314. Further, a waiting room 312 is installed toface the sidewalk 313 on the side opposite to the roadway 314. Thewaiting room 312 is surrounded by a roof and a wall, and includes a door321 for entering and leaving.

For example, the vehicle-exterior monitoring unit 171 holds in advance apattern of the shape of a facility installed at a stop, such as a sign,a bus shelter, and a waiting room, and performs pattern recognitionusing the pattern to detect a stop in the vehicle-exterior image.Alternatively, for example, the vehicle-exterior monitoring unit 171holds in advance a pattern of a mark (e.g., mark of a stop or buscompany) attached to the sign, bus shelter, or the like, and performspattern recognition using the pattern to detect a stop in thevehicle-exterior image. Alternatively, for example, the vehicle-exteriormonitoring unit 171 detects a stop by recognizing the characterrepresenting the name of the next stop by character recognition or thelike from the vehicle-exterior image.

Further, in a bus terminal or the like where buses on a plurality ofroutes arrive and depart, the stop position of the bus differs for eachroute, and in some cases, a plurality of signs 331 to 333 are installedas shown in FIG. 10, for example. In this case, the vehicle-exteriormonitoring unit 171 recognizes, from the sign 331 to the sign 333, thesign corresponding to the route of the bus 201. For example, thevehicle-exterior monitoring unit 171 recognizes the sign correspondingto the route of the bus 201 on the basis of preset information (e.g.,position of the sign in the bus terminal), or recognizes the signcorresponding to the route of the bus 201 by character recognition orthe like.

Note that the method of detecting the stop is not limited to theabove-mentioned example, and an arbitrary method can be adopted. Forexample, a stop may be detected by supplementarily using the positioninformation of the bus 201 and the map information. Alternatively, astop may be detected by performing communication with the communicationapparatus installed at the stop.

Next, the vehicle-exterior monitoring unit 171 sets the stop referenceposition on the basis of the detection result of the stop. The stopreference position represents a reference position where the bus 201stops at the next stop, and processing of recognizing a candidate forgetting on is performed for people around the stop reference position aswill be described later.

For example, in the case where a sign is installed at the stop, theinstallation position of the sign is set to the stop reference position.Further, for example, in the case where a structure (hereinafter,referred to as the stop structure) for passengers waiting for a bus suchas a bus shelter and a waiting room is installed at the stop, the stopreference position is set on the basis of the stop structure. Forexample, in the case where a bus shelter is installed, the area underthe roof of the bus shelter is set to the stop reference position.Further, for example, in the case where the getting-on position(boarding position) is indicated at the stop, the getting-on position isset to the stop reference position.

Note that in the case where the stop includes two or more of the sign,the stop structure, and (display of) the getting-on position, forexample, the stop reference position may be set on the basis of one ofthem or the stop reference position may be set on the basis of two ormore positional relationships. In the case where setting is made on thebasis of one of them, (display of) the getting-on position may beprioritized rather than the sign and the stop structure.

In Step S4, the vehicle-exterior monitoring unit 171 starts processingof recognizing a candidate for getting on. Here, the vehicle-exteriormonitoring unit 171 performs, on the area in the vicinity of the stopcandidate point, processing of recognizing a candidate for getting on.For example, the vehicle-exterior monitoring unit 171 sets the area of ahigh possibility that there is a passenger for waiting for a bus to anarea (hereinafter, referred to as the priority recognition area) onwhich the processing of recognizing a candidate for getting on ispreferentially performed, on the basis of the stop reference positionset in the processing of Step S3.

FIG. 11 shows one typical example of the priority recognition area. Inthis example, a stop reference position Pa is installed in the vicinityof the boundary between a sidewalk 401 and a roadway 402. For example,the stop reference position Pa is the installation position of the sign(not shown) of the stop. Note that on the roadway 402, a roadway outsideline 403 and a lane boundary line 404 are shown.

For example, the area within a predetermined radius around the stopreference position Pa is set to the priority recognition area. At thistime, a plurality of priority recognition areas may be set in a stepwisemanner. In the example of FIG. 11, two priority recognition areas of apriority recognition area Ala and a priority recognition area A2 a areset. For example, the priority recognition area Ala is set to a circulararea within a radius r1 a around the stop reference position Pa. Thepriority recognition area A2 a is set to a circular ring area within aradius r2 a (> radius r1 a) around the stop reference position Paexcluding the priority recognition area Ala.

The priority for performing the processing of recognizing a candidatefor getting on is the highest in the priority recognition area Ala, thesecond highest in the priority recognition area A2 a, and the lowest inthe area outside the priority recognition area A2 a. For example, themachine power, time, and the like allocated to the recognitionprocessing is the largest in the priority recognition area Ala, thesecond largest in the priority recognition area A2 a, and the lowest inthe area outside the priority recognition area A2 a. Accordingly, therecognition accuracy is the highest in the priority recognition area A1a, the second highest in the priority recognition area A2 a, and thelowest in the area outside the priority recognition area A2 a.

Note that an area outside the priority recognition area A1 b may beexcluded from the target for the processing of recognizing a candidatefor getting on. That is, only in the priority recognition area A1 a andthe priority recognition area A1 b, the processing of recognizing acandidate for getting on may be performed. In this case, a personoutside the priority recognition area A1 b is not recognized as thecandidate for getting on.

Further, for example, in the case of using a score for determiningwhether or not he/she is a candidate for getting on, a person in thepriority recognition area A1 a may be given a higher score than a personthe priority recognition area A1 b, and a person in the priorityrecognition area A1 b may be given a higher score than a person outsidethe priority recognition area A1 b.

Note that the priority recognition area does not necessarily need to beset to a concentric one. For example, as shown in FIG. 12, a priorityrecognition area A1 b and a priority recognition area A2 b may be set tohave an elliptical outer circumference whose center is the stopreference position Pa.

Further, the position where a passenger waits for a bus changesdepending on the configuration of the stop, the situation around thestop, and the like. Here, examples of the configuration of the stopinclude the presence/absence and position of a sign, thepresence/absence, shape, and position of a stop structure such as a busshelter and a waiting room, and the presence/absence and position ofdisplay of the stop position of the bus. Examples of the situationaround the stop include the position, configuration, and accessory ofthe road around the stop, the position, type, and shape of a building ormoving object around the stop, and the weather. Examples of theconfiguration of the road include the presence/absence or width of asidewalk, the presence/absence or width of a shoulder, the width of alane, and the number and position of road lane markings. Examples of theaccessory of the road include a guardrail, a fence, a curb, and a roadsign. Examples of the surrounding building include a building, a house,a wall, a fence, and a parking lot. Examples of the surrounding movingobject include a parked vehicle.

In this regard, the vehicle-exterior monitoring unit 171 may set thepriority recognition area further on the basis of at least one of theconfiguration of the stop, the situation around the stop (the stopreference position), and the like.

For example, In the stop shown in Part A of FIG. 13, a sign 421 isinstalled in the vicinity of the boundary between a sidewalk 422 and aroadway 423. A guard rail 426 is installed at the end of the sidewalk422 so as to face the roadway 423. On the roadway 423, a roadway outsideline 424 and a lane boundary line 425 are shown. A building 427 facesthe sidewalk 422.

In this case, as shown in Part B of FIG. 13, the position of the sign421 is set to a stop reference position Pc. Further, basically, an areawithin a radius r1 c around the stop reference position Pc is set to apriority recognition area A1 c, and an area within a radius r2 c (>radius r1 c) around the stop reference position Pc excluding thepriority recognition area A1 c is set to a priority recognition area A2c. However, since the possibility that a passenger waiting for a busenters the roadway 423 is assumed to be low due to the guardrail 426,the area within the roadway 423 is excluded from the priorityrecognition area. The area within the premises of the building 427 isexcluded from the priority recognition area.

FIG. 14 shows an example of the priority recognition area in a stopwhere no sidewalk is provided and a sign (not shown) is installed in ashoulder 441. Specifically, the shoulder 441 and a roadway 442 areseparated by a roadway outside line 443. On the roadway 442, a laneboundary line 444 is shown. A fence 445 is installed so as to face theshoulder 441. The sign (not shown) is installed at the end of theshoulder 441 on the side of the fence 445, and a stop reference positionPd is set to this position.

In this case, for example, basically, an area within a radius rid aroundthe stop reference position Pd is set to a priority recognition areaAid, and an area within a radius r2 d (> radius rid) around the stopreference position Pd excluding the priority recognition area Aid is setto a priority recognition area A2 d. However, the inner side (sideopposite to the shoulder 441) of the fence 445 is excluded from thepriority recognition area. Further, since the shoulder 441 is narrow andthe possibility that a passenger waits for a bus in the roadway 442 ishigh, the area within the roadway 442 is not excluded and included inthe priority recognition area.

In the stop shown in Part A of FIG. 15, a sign 461 and a bus shelter 462are installed. Specifically, the bus shelter 462 is installed at the endof a sidewalk 463 on the side of roadway 464. The sign 461 is installedat the end on the front side of the bus shelter 462. On the roadway 464,a roadway outside line 465 and a lane boundary line 466 are shown. Abuilding 467 faces the sidewalk 463.

In this case, as shown in Part B of FIG. 15, the position of the sign461 is set to a stop reference position Pe. Further, the rectangulararea in the bus shelter 462 is set to a priority recognition area Ale.Further, a rectangular frame area surrounding the priority recognitionarea Ale is set to a priority recognition area A2 e.

In this example, since the width of the sidewalk 463 is wide, thepriority recognition area A2 e is not set up to the end of the sidewalk463 on the side of the building 467. Further, since a guardrail or thelike is provided and there is a possibility a passenger waiting for abus enters the roadway 464, the area in the roadway 464 is not excludedfrom and included in the priority recognition area.

Next, the vehicle-exterior monitoring unit 171 recognizes a person inthe vicinity of the stop by face recognition of the like on the basis ofthe vehicle-exterior image. Further, the vehicle-exterior monitoringunit 171 recognizes the position, the orientation of the line of sight,the behavior, and the like of the recognized person. Then, thevehicle-exterior monitoring unit 171 extracts a candidate for getting onamong the recognized persons, on the basis of at least one of therecognition results of the position, the orientation of the line ofsight, the behavior or the like of the person in the vicinity of thestop.

Note that as the recognition processing by the vehicle-exteriormonitoring unit 171, arbitrary method can be adopted.

Now, a specific example of the processing of recognizing a candidate forgetting on with reference to FIG. 16.

FIG. 16 shows an example of the recognition result of the person aroundthe stop shown in Part A of FIG. 13. In this example, a person 501 a toa person 501 h are recognized. Note that in the figure, a frame is shownso as to surround the face of the recognized person.

The persons 501 a to 501 c stand in the vicinity of the sign 421 in thepriority recognition area A1 b. The person 501 d and the person 501 eare walking in a direction away from the stop in the priorityrecognition area A2 c. The person 501 f is riding in a bicycle in thepriority recognition area A2 c. The person 501 g is walking on thesidewalk 422 outside the priority recognition area A2 c in a directionapproaching the stop. The person 501 h is raising his/her hand whilewatching the direction of the bus 201, in the roadway 423 outside thepriority recognition area A2 c.

For example, the vehicle-exterior monitoring unit 171 excludes a personwho is presumed not to be waiting for a bus obviously from therecognized persons. For example, the persons 501 d and the person 501 eeach walking in a direction away from the stop, and the person 501 friding on a bicycle are excluded.

Next, the vehicle-exterior monitoring unit 171 gives, to each of theremaining persons, a score based on the position, the orientation of theline of sight, the behavior, and the like. For example, a person in thepriority recognition area A1 b is given a high score, and a person inthe priority recognition area A2 b is given the next highest score.Further, for example, a person running towards the stop is given a highscore, and a person walking toward the stop is given the next highestscore. Further, for example, a person watching the direction of the bus201 is given a high score. Further, a person raising his/her hand towardthe bus 201 is given a high score.

Note that for example, the vehicle-exterior monitoring unit 171 maychange the score to be given, depending on the situation around the stoprecognized by the situation recognition unit 162. For example, in thecase of a stop including a bus shelter, it is assumed that thepossibility that passengers wait in the bus shelter increases at thetime of strong sunshine or bad weather such as rain and snow as comparedwith the other cases. In this regard, at the time of strong sunshine orbad weather, the vehicle-exterior monitoring unit 171 may increase thescore to be given to a person in the bus shelter or reduce the score tobe given to a person outside the bus shelter than usual.

Then, the vehicle-exterior monitoring unit 171 extracts a person havingthe total score of not less than a predetermined threshold value as acandidate for getting on. For example, in the example of FIG. 16, theperson 501 a to person 501 c, and the person 501 h are each recognizedas a candidate for getting on.

Note that for example, the vehicle-exterior monitoring unit 171 maychange the reference for recognizing a candidate for getting on,depending on the situation around the stop recognized by the situationrecognition unit 162. For example, the vehicle-exterior monitoring unit171 may reduce the threshold value than usual, in the situation wherethe recognition accuracy of a candidate for getting on is expected todecrease. That is, the reference for recognizing a candidate for gettingon may be reduced. For example, in the case where the surroundings aredark at night, the visibility around the stop is poor due to rain, fog,snow, or the like, or the visibility around the stop is poor due to anobstacle such as another vehicle, the vehicle-exterior monitoring unit171 may reduce the threshold value than usual.

Further, for example, in the case of a stop including a waiting room asin the above-mentioned stop shown in FIG. 9, there is a possibility thatthe inside of the waiting room becomes a blind spot from the bus 201 andpassengers waiting in the waiting room cannot be recognized. In thisregard, a camera may be provided in the waiting room and an imagecaptured by the camera may be used for performing the processing ofrecognizing a candidate for getting on in the waiting room.

For example, FIG. 17 schematically shows a view of the interior of thewaiting room 312 in FIG. 9 as viewed from above. A camera 521 isinstalled in the vicinity of the ceiling of the end, which is oppositeto the door 321, of a wall 312A of the waiting room 312 on the side ofthe door 321 so that the entire interior can be imaged. Further, a bench522 is placed in the vicinity of a wall 312B facing the wall 312A.

For example, when the communication unit 25 of the bus 201 approachesthe waiting room 312, it communicates with the camera 521, and acquiresan image captured by the camera 521. Then, the vehicle-exteriormonitoring unit 171 recognizes, on the basis of the image received fromthe waiting room 312 via the communication unit 25, a person 523 a and aperson 523 b sitting on the bench 522 as candidates for getting on.

Further, the vehicle-exterior monitoring unit 171 may perform theprocessing of recognizing a candidate for getting on using all thevehicle-exterior images or perform the processing of recognizing acandidate for getting on using a part of the vehicle-exterior images.

Further, the vehicle-exterior monitoring unit 171 may change thevehicle-exterior image to be used, between before the bus 201 stops (atthe time of moving) and after the bus 201 stops (at the time ofstopping). For example, as shown in Part A of FIG. 18, in the case wherethe bus 201 approaches a stop reference position 541 set on the boundarybetween a sidewalk 542 and a roadway 543, the vehicle-exteriormonitoring unit 171 uses images captured by the camera 221F1, the camera221F2 (not shown), and the camera 221L1 for performing, on the frontdirection and the obliquely left front direction of the bus 201, theprocessing of recognizing a candidate for getting on. That is, in thecase where the bus 201 approaches the stop, the processing ofrecognizing a candidate for getting on is performed mainly on thevicinity of the stop in the travelling direction of the bus 201.

Meanwhile, as shown in Part B of FIG. 18, after the bus 201 stops at thestop, the vehicle-exterior monitoring unit 171 further adds imagescaptured by the camera 221L2 to the camera 221L4 and the camera 221B,and adds the left side of the bus 201 and the rear side of the bus 201as targets for recognizing a candidate for getting on. That is, afterthe bus 201 stops, the area to be recognized is widened so that acandidate for getting on who is around the bus 201 can be recognizedmore reliably and a passenger getting on the bus 201 and a passengergetting off the bus 201 can be recognized more reliably.

Accordingly, it is possible to more efficiently perform the processingof recognizing a candidate for getting on, the load of the recognitionprocessing is reduced, and the recognition accuracy is improved.

Note that in this case, the imaging control unit 165 may perform controlso that only a camera that captures an image used for the recognitionprocessing performs imaging and other cameras stop imaging. That is, theimaging control unit 165 may change the camera that performs imagingbetween before and after the bus 201 stops. Accordingly, it is possibleto reduce the power consumption.

Further, for example, before the bus 201 stops, the vehicle-exteriorimage to be used for the processing of recognizing a candidate forgetting on may be switched or the camera that performs imaging may beswitched, depending on the relative position between the bus 201 and thestop.

Returning to FIG. 6, in Step S5, the situation recognition unit 162determines whether or not the appointed time has passed. Here, theappointed time represents, for example, the scheduled time to depart thenext stop in the service schedule of the bus 201. In the case where itis determined that the appointed time has passed, the processingproceeds to Step S6.

In Step S6, the in-vehicle monitoring unit 172 determines whether or notthere is a passenger to get off. For example, the in-vehicle monitoringunit 172 determines, in the case where a get-off button provided in thebus 201 has been pressed, that there is a passenger to get off, anddetermines, in the case where the get-off button has not been pressed,that there is no passenger to get off. Then, in the case where it isdetermined that there is no passenger to get off, the processingproceeds to Step S7.

Note that for example, the in-vehicle monitoring unit 172 may performprocessing of recognizing a candidate for getting off to be describedlater, and determine, on the basis of the recognition result, whether ornot there is a passenger to get off.

In Step S7, the vehicle-exterior monitoring unit 171 determines, on thebasis of the result of the processing of recognizing a candidate forgetting on, whether or not there is a candidate for getting on. In thecase where it is determined that there is no candidate for getting on,the processing proceeds to Step S8.

In Step S8, the bus 201 passes the stop candidate point under thecontrol of the driving control unit 163. That is, in the case wherethere is no passenger to get off and no candidate for getting on and theappointed time has passed, the bus 201 passes the stop candidate point(next stop) without stopping.

At this time, for example, the driving control unit 163 causes the bus201 to slow down in the vicinity of the stop. Further, the UI controlunit 166 controls the voice output apparatus or the like (not shown) tooutput an announcement indicating that is passes the stop. This makes iteasier for a passenger in the bus 201 who have not noticed the approachto the stop where he/she gets off to notice the approach to the stop.Then, for example, in the case where the passenger presses the get-offbutton, processing similar to that in the case where it is determinedthat there is a passenger to get off in Step S6 to be described later isperformed.

Further, since the bus 201 slows down in the vicinity of the stop, therecognition accuracy of a candidate for getting on by thevehicle-exterior monitoring unit 171 is improved. Further, for example,in the case where a passenger at a position a little away from the stophas not noticed the approach of the bus 201, it becomes easier forhim/her to notice the approach of the bus 201. Then, the passenger takesan action for getting on, and thus, the vehicle-exterior monitoring unit171 recognizes him/her as a new candidate for getting on in some cases.In the case where the presence of a candidate for getting on isrecognized at this point, for example, processing similar to that in thecase where it is determined that there is a candidate for getting on inStep S7 to be described later is performed.

Meanwhile, in the case where a candidate for getting on and the presenceof a passenger to get off are not recognized and it has passed the stop,then, the driving control unit 163 accelerates the bus 201 up to thenormal travelling speed.

In Step S9, the vehicle-exterior monitoring unit 171 stops theprocessing of recognizing a candidate for getting on. At this time, theimaging control unit 165 may stop the imaging by the camera that is notused for applications other than the processing of recognizing acandidate for getting on.

After that, the processing returns to Step S1, and the processing ofStep S1 and subsequent Steps are executed.

Meanwhile, in the case where it is determined in Step S5 that theappointed time has not passed, there is a passenger to get off in StepS6, or it is determined in Step S7 that there is a candidate for gettingon, the processing proceeds to Step S10.

In Step S10, the vehicle control unit 151 sets a stop candidateposition. For example, the vehicle-exterior monitoring unit 171performs, in the vicinity of the stop, processing of detecting displayof the stop position of a bus, on the basis of the vehicle-exteriorimage. For example, the stop position of the bus is displayed by arectangular frame or the like on the road surface of the lane (stoplane) in front of the stop. Then, in the case where the display of thestop position is detected, the stop position setting unit 164 sets thestop position to the stop candidate position.

Further, for example, in the case where the display of the stop positionof the bus cannot be detected, the vehicle-exterior monitoring unit 171performs detection of display of the getting-on position of passengers.Then, in the case where the display of the getting-on position ofpassengers is detected, the stop position setting unit 164 sets a stopcandidate position so that the entrance door 213 of the bus 201 islocated in the vicinity of the getting-on position.

Further, for example, in the case where the display of the stop positionof the bus and the display of the getting-on position are not detected,the stop position setting unit 164 sets a stop candidate position on thebasis of the stop reference position. For example, the stop positionsetting unit 164 sets a stop candidate position so that the entrancedoor 213 of the bus 201 is located in the vicinity of the stop referenceposition.

In Step S11, the vehicle-exterior monitoring unit 171 determines whetheror not the vehicle is capable of stopping at the stop candidateposition. For example, the vehicle-exterior monitoring unit 171 detectsthe presence/absence of an obstacle in the vicinity of the stopcandidate position on the basis of the vehicle-exterior image. Here, theobstacle represents one that interferes with the stopping of the bus201, and another vehicle, a person, and the like are assumed, forexample. In the case where no obstacle is detected in the vicinity ofthe stop candidate position, the vehicle-exterior monitoring unit 171determines that the vehicle is capable of stopping at the stop candidateposition, and the processing proceeds to Step S12.

In Step S12, the bus 201 stops at the stop candidate position.Specifically, the stop position setting unit 164 sets the current stopcandidate position to the stop position. Under the control of thedriving control unit 163, the bus 201 gradually decelerates as itapproaches the stop position, and stops at the stop position. Further,under the control of the driving control unit 163, the bus 201 opens theentrance door 213 and the exit door 214. Note that in the case where nocandidate for getting on is recognized, it does not necessarily need toopen the entrance door 213. Further, in the case where there is noperson getting off, it does not necessarily need to open the exit door214. Further, in the case where the bus 201 is capable of tilting thevehicle body for assisting getting on and off, the driving control unit163 tilts the vehicle body of the bus 201. Further, the UI control unit166 controls the voice output apparatus or the like (not shown) tooutput an announcement indicating that it has stopped at the stop.

After that, the processing proceeds to Step S14.

Meanwhile, the vehicle-exterior monitoring unit 171 determines, in thecase where an obstacle is detected in the vicinity of the stop candidateposition, that the vehicle cannot stop at the stop candidate position inStep S11, and the processing proceeds to Step S13. At this time, forexample, the vehicle-exterior monitoring unit 171 may detect movement ofa moving object such as a vehicle and a person around the stop candidateposition, and determine, in the case where there is a possibility thatthe moving object enters the vicinity of the stop candidate position,that the vehicle cannot stop at the stop candidate position.

In Step S13, the bus 201 stops near the stop candidate position. Forexample, the stop position setting unit 164 sets a place where thevehicle is capable of avoiding the obstacle near the stop candidateposition in the stop lane to the stop position. Under the control of thedriving control unit 163, the bus 201 gradually decelerates as itapproaches the stop position, and stops at the stop position. Then,similarly to the processing of Step S12, the driving control unit 163opens the entrance door 213 and the exit door 214, and tilts the vehiclebody. Further, the UI control unit 166 controls the voice outputapparatus or the like (not shown) to output an announcement indicatingthat it has stopped at the stop.

After that, the processing proceeds to Step S14.

In Step S14, the in-vehicle monitoring unit 172 starts processing ofrecognizing a candidate for getting off. Specifically, the in-vehiclemonitoring unit 172 uses the in-vehicle image for recognizing persons(passengers) inside the vehicle. Further, the in-vehicle monitoring unit172 recognizes the position, behavior, and the like of each of therecognized passengers. Then, the in-vehicle monitoring unit 172extracts, on the basis of at least one of the recognition results of theposition, behavior, and the like of each of the recognized passenger, acandidate for getting off from the recognized passengers. For example,the in-vehicle monitoring unit 172 recognizes a passenger standing upfrom the seat, a passenger moving in the direction of the exit door 214,and the like as candidates for getting off.

Note that as the recognition processing by the in-vehicle monitoringunit 172, an arbitrary method can be adopted.

In Step S15, the monitoring unit 161 starts processing of recognizinggetting on (boarding) of a candidate for getting on and getting off of acandidate for getting off.

For example, the vehicle-exterior monitoring unit 171 stores the faceimage of each candidate for getting on or the feature amount of theface. Further, the vehicle-exterior monitoring unit 171 performs faceauthentication of a passenger newly getting on the bus 201, by using thestored face image of each candidate for getting on or feature amount ofthe face, the image of the vicinity of the entrance door 213 captured bythe camera 221L2, and the image captured by the in-vehicle camera 2211N.Then, in the case where there is a candidate for getting on, who has aface similar to the face of the newly getting-on passenger, thevehicle-exterior monitoring unit 171 removes the passenger from acandidate for getting on.

Further, for example, the in-vehicle monitoring unit 172 stores the faceimage of each candidate for getting off or the feature amount of theface. Further, the in-vehicle monitoring unit 172 performs faceauthentication of a passenger getting off the bus 201, by using thestored face image of each candidate for getting off or feature amount ofthe face, the image of the vicinity of the exit door 214 captured by thecamera 221L3, and the image captured by the in-vehicle camera 2211N.Then, in the case where there is a candidate for getting off, who has aface similar to the face of the getting-off passenger, thevehicle-exterior monitoring unit 171 removes the passenger from acandidate for getting off.

Note that after starting the processing of recognizing getting on of acandidate for getting on, the vehicle-exterior monitoring unit 171 maystop the processing of recognizing a candidate for getting on anddetermine a candidate for getting on once.

Alternatively, also after starting the processing of recognizing gettingon of a candidate for getting on, the vehicle-exterior monitoring unit171 may continue the processing of recognizing a candidate for gettingon and update the candidate for getting on. Accordingly, for example,the vehicle-exterior monitoring unit 171 is capable of newly recognizinga person approaching the entrance door 213 of the bus 201 late as acandidate for getting on, and removing the erroneously recognizedcandidate for getting on.

Similarly, after starting the processing of recognizing getting off of acandidate for getting off, the in-vehicle monitoring unit 172 may stopthe processing of recognizing a candidate for getting off and determinea candidate for getting off once.

Alternatively, also after starting the processing of recognizing gettingoff of a candidate for getting off, the in-vehicle monitoring unit 172may continue the processing of recognizing a candidate for getting offand update the candidate for getting off. Accordingly, for example, thein-vehicle monitoring unit 172 is capable of newly recognizing a personapproaching the exit door 214 of the bus 201 late as a candidate forgetting off, and removing the erroneously recognized candidate forgetting off.

In Step S16, the situation recognition unit 162 determines whether ornot it is before the appointed time. The determination processing ofStep S16 is repeatedly executed until it is determined to be theappointed time or it is determined that it has passed the appointedtime. Then, in the case where it is determined to be the appointed timeor it is determined that it has passed the appointed time, theprocessing proceeds to Step S17.

In Step S17, the monitoring unit 161 determines whether or not acandidate for getting on or a candidate for getting off remains. Thedetermination processing of Step S17 is repeatedly executed until it isdetermined that no candidate for getting on and no candidate for gettingoff remains. Then, in the case where it is determined that no candidatefor getting on and no candidate for getting off remains, the processingproceeds to Step S18.

Note that in the case where the processing of recognizing a candidatefor getting on is stopped after starting the processing of recognizinggetting on of a candidate for getting on, the vehicle-exteriormonitoring unit 171 may perform the processing of recognizing acandidate for getting on again just in case, after determining that nocandidate for getting on remains. Further, in the case where theprocessing of recognizing a candidate for getting on is stopped afterstarting the processing of recognizing getting on of a candidate forgetting on, the vehicle-exterior monitoring unit 171 may perform, when acandidate for getting on remains even after a predetermined time haselapsed, the processing of recognizing a candidate for getting on again,in preparation for a case where getting on of a candidate for getting oncannot be recognized due to erroneous recognition of a candidate forgetting on.

Further, in the case where the processing of recognizing a candidate forgetting off is stopped after starting the processing of recognizinggetting off of a candidate for getting off, the in-vehicle monitoringunit 172 may perform the processing of recognizing a candidate forgetting off just in case, after determining that no candidate forgetting off remains. Further, in the case where the processing ofrecognizing a candidate for getting off is stopped after starting theprocessing of recognizing getting off of a candidate for getting off,the in-vehicle monitoring unit 172 may perform, when a candidate forgetting off remains even after a predetermined time has elapsed, theprocessing of recognizing a candidate for getting off again, inpreparation for a case where getting off of a candidate for getting offcannot be recognized due to erroneous recognition of a candidate forgetting off.

Further, for example, the vehicle-exterior monitoring unit 171 mayrepeat the processing of recognizing a candidate for getting on withoutperforming the processing of recognizing getting on of a candidate forgetting on, and determine, when there is no candidate for getting on tobe recognized, that no candidate for getting on remains. Similarly, thein-vehicle monitoring unit 172 may repeat the processing of recognizinga candidate for getting off without performing the processing ofrecognizing getting off of a candidate for getting off, and determine,when there is no candidate for getting off to be recognized, that nocandidate for getting off remains.

In Step S18, the vehicle control unit 151 determines whether or not thevehicle is capable of departing. For example, the in-vehicle monitoringunit 172 detects the presence/absence of a passenger who is not seated,on the basis of the in-vehicle image. The driving control unit 163determines, in the case where a passenger who is not seated is detected,that the vehicle is not capable of departing, and determines, in thecase where a passenger who is not seated is not detected, that thevehicle is capable of departing. However, assuming a case of fulloccupancy or a case where there is a passenger standing without sitting,the driving control unit 163 determines, after a predetermined time(e.g., 30 seconds) has elapsed after no candidate for getting on and nocandidate for getting off is detected, that the vehicle is capable ofdeparting regardless of the presence/absence of the standing passenger,for example.

Then, in the case where it is determined that the vehicle is capable ofdeparting, the processing returns to Step S17, and the processing ofStep S17 and Step S18 is repeatedly executed until it is determined inStep S18 that the vehicle is capable of departing. Accordingly, a newpassenger becomes capable of getting on or getting off the vehicle untilthe vehicle is capable of departing.

Meanwhile, in the case where it is determined in Step S18 that thevehicle is capable of departing, the processing proceeds to Step S19.

In Step S19, the bus 201 departs. Specifically, in the case where thevehicle body of the bus 201 is tilted, the driving control unit 163returns the tilting of the vehicle body to its original state. Further,the driving control unit 163 closes the entrance door 213 and the exitdoor 214. Further, the UI control unit 166 controls the voice outputapparatus or the like (not shown) to output an announcement indicatingthat the vehicle departures. Then, after the vehicle-exterior monitoringunit 171 confirms the surrounding security on the basis of thevehicle-exterior image, the bus 201 departs and gradually acceleratesunder the control of the driving control unit 163.

Note that the vehicle-exterior monitoring unit 171 may perform,immediately before or immediately after the bus 201 departures,processing of recognizing a passenger who have missed the bus 201. Forexample, in the case where there is a person who runs toward the bus201, chases after the bus 201, or acts like he/she wants the bus 201 towait for departure in the image of the rear of the bus 201, thevehicle-exterior monitoring unit 171 recognizes him/her as a passengerwho has missed the bus 201. In the case where a passenger who has missedthe bus 202 is recognized and the bus 201 has already departed, thedriving control unit 163 stops the bus 201 when it is capable ofstopping. Then, the driving control unit 163 delays the departure of thebus 201 until getting on the passenger who has missed the bus 202 iscompleted.

In Step S20, the monitoring unit 161 stops the processing of recognizinga candidate for getting on and a candidate for getting off. At thistime, the imaging control unit 165 may stop the imaging by the camerathat is not used for applications other than the processing ofrecognizing a candidate for getting on or a candidate for getting off.

After that, the processing returns to Step S1, and the processing ofStep S1 and subsequent Steps is executed.

In this way, it is possible to appropriately perform automatic drivingof the bus 201, in accordance with a person getting on the bus 201 or aperson getting off the bus 201. That is, the bus 201 that performsautomatic driving is capable of reliably stopping at the stop inaccordance with the presence/absence of a passenger to get on and apassenger to get off. This allows the passenger to reliably get on thebus 201 and get off the bus 201 at a desired stop.

Further, the convenience of passengers on the bus 201 that performsautomatic driving is improved. That is, the passenger only needs to waitat a stop, and is capable of reliably getting on the bus 201 without anyspecial operation or the like.

2-3. Modified Example of First Embodiment

For example, in the case where it is determined in Step S5 that it hasnot passed the appointed time or it is determined in Step S6 that thereis a passenger to get off, i.e., the bus 201 stops regardless of thepresence/absence of a candidate for getting on, for example, thevehicle-exterior monitoring unit 171 may start, immediately before orafter the bus 201 stops, the processing of recognizing a candidate forgetting on.

Further, for example, in the case where the situation recognition unit162 recognizes a situation where the recognition accuracy of a candidatefor getting on is very low (e.g., dense fog), the bus 201 may stop atthe next stop regardless of the presence/absence of a candidate forgetting on or a passenger to get off.

Further, for example, in the case where it is recognized that there is aperson in the priority recognition area having the highest priority, thebus 201 may stop at the stop regardless of the orientation of the lineof sight, the behavior, and the like of the person.

Further, the priority recognition area can be set to one stage or threeor more stages instead of the above-mentioned two stages.

Further, in the case where the next stop is the last stop, thevehicle-exterior monitoring unit 171 does not need to perform theprocessing of recognizing a candidate for getting on. Further, forexample, in the case where the in-vehicle monitoring unit 172 recognizesthat there is no passenger in the vehicle, the bus 201 may head to thenext destination such as a garage without stopping at the last stop.

3. Second Embodiment

Next, a second embodiment of the present technology will be described.The second embodiment of the present technology is an embodiment in thecase where the vehicle 11 is a taxi that performs automatic driving andthe vehicle 11 automatically finds a passenger, stops, puts thepassenger on, and departs.

Now, automatic stop/departure processing executed by the vehicle 11 willbe described with reference to the flowchart of FIG. 19. Note that thisprocessing is started when an ACC (accessory) power source of thevehicle 11 is turned on, and finished when the ACC power source isturned off, for example.

In Step S101, the vehicle-exterior monitoring unit 171 sets the priorityrecognition area. Here, the priority recognition area represents an areaon which the processing of recognizing a candidate for getting on ispreferentially performed as described above, and an area that is apredetermined area in a predetermined direction with respect to thevehicle 11 and has a person who is likely to ride on a taxi is set tothe priority recognition area, for example.

For example, the vehicle-exterior monitoring unit 171 detects, on thebasis of the vehicle-exterior image, a sidewalk or the like on the laneside on which the vehicle 11 travels in the travelling direction of thevehicle 11. Here, the sidewalk or the like represents, for example, asidewalk or a shoulder. Note that strictly speaking, a roadway includesan area between a roadway outside line of a roadway on which a sidewalkis provided and a sidewalk (e.g., area between the sidewalk 401 and theroadway outside line 403 in FIG. 11). However, the area may be regardedas a sidewalk, because a pedestrian may pass through the area. Then, thevehicle-exterior monitoring unit 171 sets the detected sidewalk or thelike to the priority recognition area.

In Step S102, the vehicle-exterior monitoring unit 171 performs theprocessing of recognizing a candidate for getting on. For example, thevehicle-exterior monitoring unit 171 recognizes a person in the priorityrecognition area by face recognition or the like, on the basis of thevehicle-exterior image. Here, the vehicle-exterior monitoring unit 171may add the area around the priority recognition area as a recognitiontarget, or limit the recognition target to only the priority recognitionarea. Further, the vehicle-exterior monitoring unit 171 recognizes theposition, the orientation of the line of sight, the behavior, and thelike of the recognized person. Then, the vehicle-exterior monitoringunit 171 extracts, from the recognized persons, a candidate for gettingon, on the basis of at least one of recognitions results of theposition, the orientation of the line of sight, the behavior, and thelike of the recognized person.

For example, the vehicle-exterior monitoring unit 171 excludes a personwho is presumed not to be waiting for a bus obviously from therecognized persons. For example, a walking person, a person riding on abicycle, and the like are excluded.

Next, the vehicle-exterior monitoring unit 171 gives, to each of theremaining persons, a score based on the position, the orientation of theline of sight, the behavior, and the like. For example, a person in thepriority recognition area, a person raising his/her hand, a personturning his/her line of sight in the direction of the vehicle 11, or thelike is given a high score. Then, the vehicle-exterior monitoring unit171 sets a person having the total score of not less than apredetermined threshold value to a candidate for getting on. At thistime, the candidate for getting on may include two or more persons.

Note that for example, the vehicle-exterior monitoring unit 171 mayperform the above-mentioned processing of recognizing a candidate forgetting on the bus, and exclude the candidate for getting on the bus torecognize the candidate for getting on the taxi.

In Step S103, the vehicle-exterior monitoring unit 171 determineswhether or not there is a candidate for getting on, on the basis of theresult of processing of Step S102. In the case where it is determinedthat there is no candidate for getting on, the processing returns toStep S101.

After that, the processing of Step S101 to Step S103 is repeatedlyexecuted until it is determined in Step S103 that there is a candidatefor getting on.

Meanwhile, in the case where it is determined in Step S103 that there isa candidate for getting on, the processing proceeds to Step S104.

In Step S104, under the control of the driving control unit 163, thevehicle 11 approaches a candidate for getting on. At this time, thevehicle 11 slows down as necessary. Further, in the case where there area plurality of candidates for getting on, the vehicle 11 approaches anyof the plurality of candidates for getting on. For example, the vehicle11 approaches the nearest candidate for getting on or candidate forgetting on having the highest score.

In Step S105, the vehicle-exterior monitoring unit 171 specifies acandidate for getting on. For example, the vehicle-exterior monitoringunit 171 performs the processing of recognizing a candidate for gettingon again. Then, the vehicle-exterior monitoring unit 171 specifies oneperson as a candidate for getting on from persons having the total scoreof not less than the predetermined threshold value. For example, thevehicle-exterior monitoring unit 171 specifies, as a candidate forgetting on, the nearest person or the person having the highest scoreout of persons having the total score of not less than the predeterminedthreshold value. Further, in the case where there is a person near thespecified candidate for getting on, who is estimated to accompany thespecified candidate for getting on, the vehicle-exterior monitoring unit171 adds the person to the candidate for getting on.

In Step S106, the vehicle-exterior monitoring unit 171 determines, onthe basis of the result of the processing of Step S105, whether or not acandidate for getting on has been able to be specified. In the casewhere it is determined that a candidate for getting on has been able tobe specified, e.g., there is no person having the total score of notless than the predetermined threshold value, the processing returns toStep S1. Note that at this time, in the case where the vehicle 11 isslowing down, it accelerates up to the normal travelling speed.

After that, the processing of Step S101 to Step S106 is repeatedlyexecuted until it is determined in Step S106 that a candidate forgetting on has been able to be specified.

Meanwhile, in the case where it is determined in Step S106 that acandidate for getting on has been able to be specified, the processingproceeds to Step S107.

In Step S107, under the control of the driving control unit 163, thevehicle 11 sends a signal to a candidate for getting on. For example,the vehicle 11 uses headlight flashing toward a candidate for getting onor sounds a car horn (klaxon).

Note that the processing of Step S107 may be omitted.

In Step S108, the vehicle-exterior monitoring unit 171 determineswhether or not the vehicle is capable of stopping near the candidate forgetting on. For example, the vehicle-exterior monitoring unit 171detects, on the basis of the vehicle-exterior image, thepresence/absence of an obstacle around the candidate for getting on.Here, the obstacle represents one that interferes with the stopping ofthe vehicle 11, and another vehicle, a person, and the like are assumed,for example. Further, for example, the vehicle-exterior monitoring unit171 detects the presence/absence of an area where the vehicle cannotstop around the candidate for getting on. Here, the area where thevehicle cannot stop represents, for example, an area where stopping isexplicitly prohibited (e.g., a no-stopping/parking area and a dedicatedbus lane), a vicinity of an intersection, a pedestrian crossing, or thelike.

In the case where there is no obstacle and no area where the vehiclecannot stop around the candidate for getting on, the vehicle-exteriormonitoring unit 171 determines that the vehicle is capable of stoppingnear the candidate for getting on, and the processing proceeds to StepS109.

In Step S109, the vehicle 11 stops near the candidate for getting on.For example, the stop position setting unit 164 sets a stop positionnear the candidate for getting on. Under the control of the drivingcontrol unit 163, the vehicle 11 gradually decelerates as it approachesthe stop position, and stops at the stop position. After that, thevehicle-exterior monitoring unit 171 confirms, on the basis of thevehicle-exterior image, the surrounding security, and then, the drivingcontrol unit 163 opens the door of the rear seat of the vehicle 11.Further, the driving control unit 163 opens the door of the passenger'sseat and the door of the trunk of the vehicle 11 as necessary.

After that, the processing proceeds to Step S111.

Meanwhile, in the case where there is an obstacle or an area where thevehicle cannot stop around the candidate for getting on in Step S108,the vehicle-exterior monitoring unit 171 determines that the vehiclecannot stop near the candidate for getting on, and the processingproceeds to Step S110.

In Step S110, the vehicle 11 stops at a place a little away from thecandidate for getting on. For example, the vehicle-exterior monitoringunit 171 detects a place where the vehicle 11 can stop, which is closestto the candidate for getting on, and the stop position setting unit 164sets the place to the stop position. Under the control of the drivingcontrol unit 163, the vehicle 11 gradually decelerates as it approachesthe stop position, and stops at the stop position. After that, thevehicle-exterior monitoring unit 171 confirms, on the basis of thevehicle-exterior image, the surrounding security, and then, the drivingcontrol unit 163 opens the door of the rear seat of the vehicle 11.Further, the driving control unit 163 opens the door of the passenger'sseat and the door of the trunk of the vehicle 11 as necessary.

Note that at this time, for example, in the case where the set stopposition is too far from the candidate for getting on, e.g., thedistance between the set stop position and the candidate for getting onis not less than a predetermined threshold value, the vehicle 11 maypass as it is without stopping.

After that, the processing proceeds to Step S111.

Note that in the case where it is difficult for the vehicle 11 to stopat the set stop position due to the relationship or the like with asurrounding vehicle (e.g., following vehicle) in Step S109 and StepS110, it may pass as it is without stopping.

In Step S111, the vehicle-exterior monitoring unit 171 starts processingof recognizing getting on of a candidate for getting on. For example,the vehicle-exterior monitoring unit 171 stores the face image of eachcandidate for getting on or the feature amount of the face. Further, thevehicle-exterior monitoring unit 171 performs face authentication of apassenger newly getting on the vehicle 11, by using the stored faceimage of each candidate for getting on or feature amount of the face,the vehicle-exterior image, and the in-vehicle image. Then, in the casewhere there is a candidate for getting on, who has a face similar to theface of the newly getting-on passenger, the vehicle-exterior monitoringunit 171 removes the passenger from a candidate for getting on.

In Step S112, the driving control unit 163 determines whether or not thevehicle is capable of departing. For example, in the case where thevehicle-exterior monitoring unit 171 has not recognized getting on ofall the candidates for getting on, the driving control unit 163determines that the vehicle cannot depart, and the processing proceedsto Step S113.

In Step S113, the vehicle-exterior monitoring unit 171 determineswhether or not erroneous recognition of a candidate for getting onoccurs. In the case where it is determined that erroneous recognition ofa candidate for getting on has not occurred, the processing returns toStep S112.

After that, the processing of Step S112 and Step S113 is repeatedlyexecuted until it is determined in Step S112 that the vehicle is capableof departing or it is determined in Step S113 that erroneous recognitionof a candidate for getting on has occurred.

Meanwhile, in the case where the vehicle-exterior monitoring unit 171has recognized getting on of all the candidates for getting on in StepS112, the driving control unit 163 determines that the vehicle iscapable of departing, and the processing proceeds to Step S114.

Further, for example, in the case where a candidate for getting on doesnot start getting on even when a predetermined time T1 has elapsed afterthe vehicle stops in Step S113, the vehicle-exterior monitoring unit 171determines that erroneous recognition of a candidate for getting on hasoccurred, and the processing proceeds to Step S114.

Further, for example, in the case where getting on of all the candidatesfor getting on cannot be recognized even when a predetermined time T2 (>the time T1) has elapsed after the vehicle stops, the vehicle-exteriormonitoring unit 171 determines the erroneous recognition of a candidatefor getting on has occurred, and the processing proceeds to Step S114.Note that the time T2 is set to be longer than a standard time necessaryfor getting on, considering the time or the like taken for the passengerto put the luggage in the trunk.

In Step S114, the vehicle 11 departs. Specifically, the driving controlunit 163 closes the door and the trunk of the vehicle 11. Then, thevehicle-exterior monitoring unit 171 confirms the surrounding securityon the basis of the vehicle-exterior image, and then, the vehicle 11departs and gradually accelerates under the control of the drivingcontrol unit 163.

After that, the automatic stop/departure processing is finished.

In this way, it is possible to appropriately perform automatic drivingof a taxi in accordance with the passenger riding on the taxi (thevehicle 11). That is, the taxi that performs automatic driving iscapable of automatically finding a passenger, stopping, putting thepassenger on, and departing.

Further, the convenience of passengers on the taxi that performsautomatic driving is improved. That is, the passenger only needs toperform the same operation as that in the case of riding on a taxi thatdoes not perform automatic driving, and is capable of riding on a taxithat performs automatic driving.

Note that this automatic stop/departure processing is applicable to notonly a taxi but also a moving object that passengers are capable ofgetting on at arbitrary places, which performs automatic driving in atransportation system. For example, a route bus or the like thatperforms automatic driving is assumed.

4. Third Embodiment

Next, a third embodiment of the present technology will be described.The third embodiment of the present technology is an embodiment in thecase where the vehicle 11 is a taxi that performs automatic driving andthe vehicle 11 automatically finds, when a candidate for getting on hasmade a reservation of dispatch by using application software thatdispatches a taxi (hereinafter, referred to as the dispatch APP), forexample, the candidate for getting on, stops, puts the candidate on, anddeparts.

Here, the automatic stop/departure processing performed by the vehicle11 will be described with reference to the flowchart of FIG. 20. Notethat this processing is started when, for example, a candidate forgetting on makes a reservation of taxi dispatch by using the dispatchAPP in a portable terminal apparatus such as a smartphone, a tablet, anda mobile phone, and transmits information indicating the reservationcontent thereof (hereinafter, referred to as the dispatch reservationinformation), and the communication unit 25 of the vehicle 11 receivesthe dispatch reservation information via a network, a server, and thelike.

The dispatch reservation information includes, for example, the currentposition of the candidate for getting on, the planned getting-onposition, the planned getting-on time, and the planned number of personsto get on. Note that the current position of the candidate for gettingon may be replaced with GPS (Global Positioning System) informationreceived by GPS receiver provided in a portable terminal apparatus of acandidate for getting on. Further, in the case where the dispatchreservation information does not include the planned getting-onposition, the current position of the candidate for getting on may beregarded as the planned getting-on position. Further, the dispatchreservation information may include, for example, a face image of thecandidate for getting on or face feature amount data indicating thefeature amount of the face of the candidate for getting on.

In Step S151, under the control of the driving control unit 163, thevehicle 11 moves to the planned getting-on position. At this time, inthe case where the planned getting-on time is designated, the vehicle 11moves to the planned getting-on position so as to be in time for theplanned getting-on time.

In Step S152, the vehicle-exterior monitoring unit 171 performs theprocessing of recognizing a candidate for getting on. For example, inthe case where the dispatch reservation information does not include theface image or face feature amount data of the candidate for getting on,the vehicle-exterior monitoring unit 171 performs the processing ofrecognizing a candidate for getting on, by processing similar to that ofStep S102 in FIG. 19.

Meanwhile, in the case where the dispatch reservation informationincludes the face image or face feature amount data of the candidate forgetting on, the vehicle-exterior monitoring unit 171 recognizes a personin the vehicle-exterior image. Then, the vehicle-exterior monitoringunit 171 recognizes a candidate for getting on, by checking (faceauthentication) the face image or face feature amount of the recognizedperson against the face image or face feature amount of a candidate forgetting on included in the dispatch reservation information.

Note that the processing of recognizing a candidate for getting on isrepeatedly executed a predetermined number of times or for apredetermined time even in the case where recognition of a candidate forgetting on fails. Further, in the case where the planned getting-on timeis set in the dispatch reservation information, the processing ofrecognizing a candidate for getting on is repeatedly executed until theplanned getting-on time or a predetermined time after the plannedgetting-on time even in the case where recognition of a candidate forgetting on fails.

In Step S153, the vehicle-exterior monitoring unit 171 determines, onthe basis of the result of the processing of Step S152, whether or not acandidate for getting on has been able to be specified. In the casewhere it is determined that a candidate for getting on has been able tobe specified, the processing proceeds to Step S154.

In Step S154, the vehicle 11 notifies the candidate for getting on. Forexample, the vehicle 11 notifies the candidate for getting on by sendinga signal to the candidate for getting on, similarly to the processing ofStep S107 in FIG. 19. Alternatively, for example, the communication unit25 transmits a message notifying that it has arrived at the plannedgetting-on position to a portable terminal apparatus of the candidatefor getting on via a network, a server, and the like.

After that, processing similar to that of Step S108 to Step S114 in FIG.19 is executed in Step S155 to Step S161, and the automaticstop/departure processing is finished.

Note that in Step S157, the vehicle 11 always stops even at a place awayfrom the candidate for getting on unlike the above-mentioned processingof Step S110 in FIG. 19.

Further, in Step S158, in the case where the dispatch reservationinformation include the planned number of persons to get on, thevehicle-exterior monitoring unit 171 may perform processing ofrecognizing getting on of a candidate for getting on, by comparing thenumber of persons who have got on and the planned number of person toget on.

Meanwhile, in the case where it is determined in Step S153 that acandidate for getting on has not been able to be specified, theprocessing of Step S154 to Step S161 is omitted, and the automaticstop/departure processing is finished. That is, stopping and departureof the vehicle 11 are not performed.

Note that in the case where a candidate for getting on has not been ableto be specified, for example, the communication unit 25 may transmit amessage notifying that a candidate for getting on has not been able tobe found to a portable terminal apparatus of the candidate for gettingon via a network, a server, and the like. After that, the vehicle 11 mayspecify a candidate for getting on while performing communication withthe candidate for getting on.

This automatic stop/departure processing is applicable to not only ataxi but also a case of automatically picking up a reserved/registeredcandidate for getting on.

For example, it is applicable to a case where a private car thatperforms automatic driving automatically picks up a reserved/registeredcandidate for getting on. For example, an owner or the like of theprivate car registers the face image or face feature amount data of acandidate for getting on to be picked up, in the vehicle 11 or a servercapable of communicating with the vehicle 11 in advance. Note that asthe candidate for getting on, for example, the owner of the vehicle 11,and the family member, relative, acquaintance, friend, and the like ofthe owner are assumed. Then, the vehicle 11 recognizes the registeredcandidate for getting on by the above-mentioned processing using theface image or face feature amount data, and picks up the recognizedcandidate for getting on.

Further, for example, also in the case of a route bus, a reservation ofgetting on may be made in advance, the face image or face feature amountdata may be transmitted to the bus or a server or the like capable ofcommunicating with the bus, and the bus may perform the processing ofrecognizing a candidate for getting on by using face authentication.

5. Fourth Embodiment

Next, a fourth embodiment of the present technology will be described.

Note that in the following description, unless otherwise distinguished,the vehicle 11 includes the above-mentioned bus 201.

Although the case where the present technology is applied to the casewhere the vehicle 11 performs automatic driving has been described aboveas an example, the present technology is applicable to also a case wherethe vehicle 11 performs semi-automatic driving.

For example, the vehicle 11 may automatically stop as described above,and depart by the driver's operation.

Further, for example, in the case where a candidate for getting on hasbeen recognized, the vehicle 11 may automatically decelerate and thenslow down, and the final stop decision may be made by the driver.

Further, for example, in the case where the vehicle 11 performssemi-automatic driving, a screen that notifies the recognition result orthe like of a candidate for getting on (hereinafter, referred to as theassist screen) may be presented to the driver to assist the driving. Theassist screen is displayed on the display unit 24 of the vehicle 11under the control of the UI control unit 166, for example.

FIG. 21 to FIG. 24 schematically shows an example of the assist screen.

FIG. 21 shows an example of the assist screen presented to the driver inthe case where a bus approaches a stop.

On the assist screen in FIG. 21, some pieces of information aresuperimposed on the image of the front of the bus. Specifically, a frame601 indicating the position of a sign of a stop is displayed. A frame602 indicating the priority recognition area is displayed. A frame 603indicating the stop position is displayed. Frames 604 a to 604 dsurrounding faces of recognized persons are displayed. Note that bychanging the design, color, or the like of the frames 604 a to 604 dbetween a candidate for getting on and other persons, the candidate forgetting on and other persons may be distinguished with each other.

Further, at the bottom of the screen, a message 605 indicating thatthere is a candidate for getting on is displayed. At the upper leftcorner of the screen, a route map 606 of the bus is displayed. At thecentral upper end of the screen, a current time 607 is displayed. On theright side of the current time 607, a remaining time 608 to the timewhen it should arrive at the stop is countdown-displayed. At the upperright corner of the screen, a numerical number 609 indicating the numberof persons recognized around the stop is displayed. Under the numericalnumber 609, a numerical number 610 indicating the number of recognizedcandidates for getting on is displayed. Under the numerical number 609,a message 611 indicating the presence/absence of a passenger to get offis displayed. Under the message 611, a numerical number 612 indicatingnumber of vacant seats is displayed.

With this assist screen, the driver is capable of reliably recognizingthe stop and the stop position. Further, the driver is capable ofreliably recognizing the presence/absence, number, and positions ofcandidates for getting on. Further, the driver is capable of reliablyrecognizing the presence/absence of a person to get off and the numberof vacant seats.

FIG. 22 shows an example of the assist screen presented to the driverimmediately after the bus departs the stop.

On the assist screen in FIG. 22, the image of the rear of the bus isdisplayed, and some pieces of information are superimposed on thedisplayed image. Specifically, a frame 631 surrounding a candidate forgetting on, who is recognized as a passenger who has missed the busbecause he/she is running toward the bus, is displayed. At the upperright corner of the screen, a mark 632 for calling attention to thedriver is displayed. At the lower center of the screen, a message 633indicating the there is a passenger who has missed the bus is displayed.

Accordingly, the driver is capable of reliably recognizing that there isa passenger who has missed the bus.

Note that for example, in addition to the assist screen in FIG. 22, itmay be possible to notify that there is a passenger who has missed thebus by sound such as an alarm. Further, for example, under the controlof the driving control unit 163, the vehicle 11 may automatically slowdown to urge the driver to stop.

FIG. 23 shows an example of the assist screen presented to the driver ina bus or taxi that passengers are capable of getting on at arbitraryplaces.

On the assist screen in FIG. 23, the image of the front of the vehicleis displayed, and some pieces of information are superimposed on thedisplayed image. Specifically, a frame 651 surrounding a candidate forgetting on, who has been recognized because he/she is raising his/herhand facing the vehicle, is displayed. At the upper right corner of thescreen, a mark 652 for calling attention to the driver is displayed. Atthe lower center of the screen, a message 653 indicating that acandidate for getting has been recognized on is displayed.

Accordingly, the driver is capable of reliably recognizing that there isa candidate for getting on.

Note that for example, in addition to the assist screen in FIG. 23, itmay be possible to notify that there is a candidate for getting on bysound such as an alarm. Further, for example, under the control of thedriving control unit 163, the vehicle 11 may automatically slow down tourge the driver to stop.

FIG. 24 shows an example of the assist screen presented to the driver ofa taxi.

On the assist screen in FIG. 24, some pieces of information aresuperimposed on the image of the front of the taxi. Specifically, aframe 671 a and a frame 671 b surrounding the whole body of candidatesfor getting on out of persons recognized in the priority recognitionarea are displayed. Further, a frame 672 a and a frame 672 b surroundingfaces of persons other than the candidates for getting on out of personsrecognized in the priority recognition area are displayed. Further, anarrow 673 indicating the position of the candidate for getting on isdisplayed. Accordingly, it is possible to easily recognizing thecandidate for getting on from the assist screen.

Note that since persons walking on a pedestrian crossing are locatedaway from the priority recognition area and not recognized, a frame forthe persons is not displayed.

Further, a frame 674 indicating the stop position is displayed on theroad surface before the candidates for getting on. Accordingly, thedriver is capable of stopping the vehicle 11 at a position suitable forpicking up the candidates for getting on.

Further, at the upper right corner of the screen, a map 675 isdisplayed. Under the map 675, a numerical number 676 indicating thenumber of candidates for getting on is displayed.

With this assist screen, the driver is capable of reliably recognizingthe presence of a candidate for getting on. Further, the driver iscapable of reliably recognizing the number and position of candidatesfor getting on.

Note that for example, in the case where the display unit 24 is atransmissive display superimposed and provided on a windshield part, forexample, the assist screen in FIG. 23 and FIG. 24 can be realized bysuperimposing information on the field of view (e.g., real world outsidethe vehicle 11 seen from the windshield) of an occupant of the vehicle11 by AR (augmented reality) or the like.

Further, the assist screen in FIG. 21 to FIG. 24 is applicable to avehicle that does not perform automatic driving and semi-automaticdriving.

6. Modified Example

Hereinafter, a modified example of the above-mentioned embodiment of thetechnology according to the present disclosure will be described.

For the processing of recognizing a candidate for getting on, an imagecaptured at a place other than that of the vehicle 11, such as the imagecaptured by the camera of the waiting room of the stop described above,can be used. For example, an image captured by a camera provided in asign of the stop, a surveillance camera provided in an urban area, orthe like is assumed. Further, for example, the processing of recognizinga candidate for getting on may be performed by a camera outside thevehicle 11 or a system including the camera, and the vehicle 11 mayacquire the result of the recognition processing.

Further, it is desirable to delete the face image or face feature amountdata of the passenger used by the vehicle 11, for privacy protection.For example, immediately after getting on or getting off of a passengeris recognized, the face image or face feature amount data of thepassenger may be deleted.

Alternatively, for example, after a predetermined time has elapsed fromwhen getting on or getting off of a passenger is recognized, the faceimage or face feature amount data of the passenger may be deleted.Accordingly, for example, in the case where the passenger has come topick up the lost item or the like, it is easy to confirm whether or notthe passenger has actually gotten on.

Further, although the example in which the bus 201 performs both of theprocessing of recognizing a candidate for getting on and the processingof recognizing a candidate for getting off has been described above,only one of them may be performed. Similarly, the bus 201 may performonly one of the processing of recognizing getting on of a candidate forgetting on and the processing of recognizing getting off of a candidatefor getting off.

Further, an example in which the vehicle 11 that is a taxi performs theprocessing of recognizing a candidate for getting off and the processingof recognizing a candidate for getting off has not been described above,these processes may be performed. This is effective, for example, whenthe getting-off position differs for each passenger as in a route taxi.

Further, the vehicle 11 may determine, in the case where the passengergets off the vehicle 11, whether or not the getting off is temporalgetting off, and wait, in the case of determining that it is temporalgetting off, without departing until the passenger gets on again.

Note that as the determination of whether or not it is temporal gettingoff, an arbitrary method can be adopted. For example, each passengerregisters a destination in advance by using an application program orthe like. Then, the monitoring unit 161 performs face authentication ofthe passenger who has gotten off, and recognizes the destination of thepassenger. Then, for example, the monitoring unit 161 may determine, inthe case where the place where the passenger has gotten off is differentfrom the destination of the passenger, that it is temporal getting off.Alternatively, for example, the monitoring unit 161 may determine, inthe case where the destination is such a place that the passengertemporarily stops by (e.g., a service area or parking area of a highway,and convenience store), that it is temporal getting off.

Further, for example, in the case where the vehicle-exterior monitoringunit 171 detects an obstacle in the travelling direction of the vehicle11 and the passenger gets off after the vehicle 11 stops, the passengeris assumed to have gotten off in order to remove the obstacle. In thisregard, the driving control unit 163 may try not to depart again untilthe vehicle-exterior monitoring unit 171 recognizes that the obstaclehas been removed and the passenger gets off again.

Note that the present technology is applicable also to a moving objectthat performs automatic driving or semi-automatic driving other than avehicle.

For example, the present technology is applicable to a moving objectused in a transportation system where stop points for passengers to geton and get off are determined. In particular, the present technology isapplicable to a moving object used in a transportation system where itpasses the stop point depending on the presence/absence of a passengerwaiting at the stop point and the presence/absence of a passenger to getoff at the stop point, such as a bus. For example, as such a movingobject, a train, a water bus, or the like is assumed.

Further, for example, the present technology is applicable to a movingobject used in a transportation system where passengers are capable ofgetting on at arbitrary places, such as a taxi. Further, for example,the present technology is applicable also to a private moving objectcapable of picking up an occupant at an arbitrary place, such as theabove-mentioned private car.

7. Others 7-1. Configuration Example of Computer

The series of processes described above can be performed by hardware orsoftware. In the case where the series of processes are performed by thesoftware, programs that constitute the software are installed in acomputer incorporated in dedicated hardware (e.g., processor or the likeof various ECUs).

It should be noted that the program executed by the computer may be aprogram, the processes of which are performed in a chronological orderalong the description order in the specification, or may be a program,the processes of which are performed in parallel or at necessary timingswhen being called, for example. Alternatively, a plurality of computersmay cooperate with each other to perform the above-mentioned processes.One or more computers that perform the above-mentioned processesconstitute the computer system.

Note that herein, the system refers to a set of a plurality ofcomponents (apparatuses, modules (parts), and the like). Whether all thecomponents are in the same casing or not is not considered. Therefore,both of a plurality of apparatuses stored in separate casings andconnected via a network and one apparatus having a plurality of modulesstored in one casing are systems.

Further, the embodiments of present technology are not limited to theabove-mentioned embodiments and can be variously modified withoutdeparting from the essence of the present technology.

For example, the present technology can have the configuration of cloudcomputing in which one function is shared by a plurality of apparatusesvia a network and processed in cooperation with each other.

Further, the steps described in the flowchart described above can beexecuted by one apparatus or by a plurality of apparatuses in a sharingmanner.

Further, in the case where one step includes a plurality of processes,the plurality of processes in the one step can be performed by oneapparatus or shared by a plurality of apparatus.

7-2. Configuration Combination Examples

The present technology may also take the following configurations.

(1) A moving object control apparatus, including

a driving control unit that performs, on a basis of recognition of atleast one of a candidate for getting on a moving object or a candidatefor getting off the moving object, acceleration/deceleration control ofthe moving object.

(2) The moving object control apparatus according to (1) above, in which

the driving control unit performs, on a basis of at least one of therecognition of the candidate for getting on or the recognition of thecandidate for getting off, stop control of the moving object.

(3) The moving object control apparatus according to (1) or (2) above,in which

the driving control unit performs, on a basis of at least one ofrecognition of getting on the moving object of the candidate for gettingon or recognition of getting off the moving object of the candidate forgetting off, starting control of the moving object.

(4) The moving object control apparatus according to (3) above, in which

the recognition of the candidate for getting on is performed on an areain a vicinity of a predetermined stop candidate point of the movingobject.

(5) The moving object control apparatus according to (4) above, in which

a priority recognition area is set on a basis of a predeterminedreference position in the stop candidate point, the candidate forgetting on being preferentially recognized in the priority recognitionarea.

(6) The moving object control apparatus according to (5) above, in which

the reference position is at least one of a sign indicating the stopcandidate point, a predetermined structure at the stop candidate point,or a predetermined boarding position at the stop candidate point.

(7) The moving object control apparatus according to (5) or (6) above,in which

the priority recognition area is further set on a basis of at least oneof a configuration of the stop candidate point or a situation around thestop candidate point.

(8) The moving object control apparatus according to any one of (5) to(7) above, in which

the reference position is changed depending on a situation around thestop candidate point.

(9) The moving object control apparatus according to any one of (4) to(8) above, in which

the area to be recognized is set to a predetermined area in apredetermined direction with respect to the moving object.

(10) The moving object control apparatus according to any one of (1) to(9), in which

the recognition of the candidate for getting on is performed on a basisof at least one of a face, an orientation of a line of sight, aposition, or behavior of a person recognized in an image captured by animaging unit of the moving object.

(11) The moving object control apparatus according to (10) above, inwhich

the recognition of the candidate for getting on is further performed ona basis of an image captured by an imaging apparatus outside the movingobject.

(12) The moving object control apparatus according to any one of (1) to(11) above, in which

the recognition of the candidate for getting on or the candidate forgetting off is performed on a basis of face authentication using animage of a face of the candidate for getting on or the candidate forgetting off or data indicating a feature amount of the face, the imageand the data being acquired in advance.

(13) The moving object control apparatus according to any one of (1) to(12) above, further including

a plurality of imaging units for recognizing the candidate for gettingon or the candidate for getting off in different directions or differentpositions in the moving object, in which

the imaging unit used for processing of recognizing the candidate forgetting on or the candidate for getting off is changed between when themoving object moves and when the moving object stops.

(14) The moving object control apparatus according to any one of (1) to(13) above, in which

the recognition of the candidate for getting off is performed on a basisof at least one of a position or behavior of an occupant of the movingobject in an image obtained by imaging an inside of the moving object.

(15) The moving object control apparatus according to any one of (1) to(14) above, further including

a user interface control unit that controls notification of arecognition result of the candidate for getting on.

(16) The moving object control apparatus according to (15) above, inwhich

the user interface control unit superimposes, on an image captured by animaging unit of the moving object, information indicating therecognition result of the candidate for getting on, and displays theimage.

(17) The moving object control apparatus according to (15) or (16)above, in which

the user interface control unit superimposes, on a field of view of anoccupant of the moving object, information indicating the recognitionresult of the candidate for getting on, for display.

(18) A moving object control method, including

a driving control step of performing, on a basis of recognition of atleast one of a candidate for getting on a moving object or a candidatefor getting off the moving object, acceleration/deceleration control ofthe moving object.

(19) A moving object, including

a driving control unit that performs, on a basis of recognition of atleast one of a candidate for getting on or a candidate for getting off,acceleration/deceleration control.

REFERENCE SIGNS LIST

-   -   11 vehicle    -   21 front sensing camera    -   22 front camera ECU    -   23 position information acquisition unit    -   24 display unit    -   25 communication unit    -   27 radar    -   28 lidar    -   29, 29L, 29R side view camera    -   30, 30L, 30R side view camera ECU    -   31 integrated ECU    -   32 front view camera    -   33 front view camera ECU    -   34 braking device    -   39 rear view camera    -   40 rear view camera ECU    -   42 in-vehicle camera    -   43 in-vehicle camera ECU    -   51 driving control ECU    -   101 image composition ECU    -   151 vehicle control unit    -   161 monitoring unit    -   162 situation recognition unit    -   163 travelling control unit    -   164 stop position recognition unit    -   165 imaging control unit    -   166 UI control unit    -   171 vehicle-exterior monitoring unit    -   172 in-vehicle monitoring unit    -   201 bus    -   221F1 to 221B camera    -   Pa to Pe stop reference position    -   A1 a to A2 e priority recognition area    -   521 camera

The invention claimed is:
 1. A vehicle control apparatus, comprising: acontroller configured to control acceleration or deceleration of avehicle on a basis of recognition, while the vehicle is moving, of atleast one of a person who is outside the vehicle as a candidate forgetting on the vehicle or a person who is a passenger on the vehicle asa candidate for getting off the vehicle, wherein the recognition of thecandidate for getting on the vehicle is performed within a priorityrecognition area in a vicinity of a stop of the vehicle, the stop beinga scheduled location for getting on the vehicle or off the vehicle, thepriority recognition area being set on a basis of a predeterminedreference position in the vicinity of the stop.
 2. The vehicle controlapparatus according to claim 1, wherein the controller is furtherconfigured to control stopping of the vehicle on a basis of at least oneof the recognition of the candidate for getting on the vehicle or therecognition of the candidate for getting off the vehicle.
 3. The vehiclecontrol apparatus according to claim 1, wherein the controller isfurther configured to control starting of the vehicle on a basis of atleast one of recognition of the candidate for getting on the vehiclegetting on the vehicle or recognition of the candidate for getting offthe vehicle getting off the vehicle.
 4. The vehicle control apparatusaccording to claim 1, wherein the reference position is at least one ofa sign indicating the stop, a predetermined structure at the stop, or apredetermined boarding position at the stop.
 5. The vehicle controlapparatus according to claim 1, wherein the priority recognition area isfurther set on a basis of at least one of a configuration of the stop ora situation around the stop.
 6. The vehicle control apparatus accordingto claim 1, wherein the reference position is changed depending on asituation around the stop.
 7. The vehicle control apparatus according toclaim 1, wherein the priority recognition area is set to a predeterminedarea in a predetermined direction with respect to the vehicle.
 8. Thevehicle control apparatus according to claim 1, wherein the recognitionof the candidate for getting on the vehicle is performed on a basis ofat least one of a face, an orientation of a line of sight, a position,or behavior of a person recognized in an image captured by a camera ofthe vehicle.
 9. The vehicle control apparatus according to claim 8,wherein the recognition of the candidate for getting on the vehicle isfurther performed on a basis of an image captured by a camera outsidethe vehicle.
 10. The vehicle control apparatus according to claim 1,wherein the recognition of the candidate for getting on the vehicle orthe candidate for getting off the vehicle is performed on a basis offace authentication using an image of a face of the candidate forgetting on the vehicle or the candidate for getting off the vehicle ordata indicating a feature amount of the face, the image and the databeing acquired in advance.
 11. The vehicle control apparatus accordingto claim 1, further comprising: a plurality of cameras configured toimage the candidate for getting on the vehicle or the candidate forgetting off the vehicle in different directions or different positionsin the vehicle, wherein a camera configured to image the candidate forgetting on the vehicle or the candidate for getting off the vehicle ischanged between when the vehicle moves and when the vehicle stops. 12.The vehicle control apparatus according to claim 1, wherein therecognition of the candidate for getting off the vehicle is performed ona basis of at least one of a position or behavior of an occupant of thevehicle in an image obtained by imaging an inside of the vehicle. 13.The vehicle control apparatus according to claim 1, further comprising:a user interface controller configured to control notification of arecognition result of the candidate for getting on the vehicle.
 14. Thevehicle control apparatus according to claim 13, wherein the userinterface controller is configured to superimpose, on an image capturedby a camera of the vehicle, information indicating the recognitionresult of the candidate for getting on the vehicle, and display theimage.
 15. The vehicle control apparatus according to claim 13, whereinthe user interface controller is configured to superimpose, on a fieldof view of an occupant of the vehicle, information indicating therecognition result of the candidate for getting on the vehicle, fordisplay.
 16. A vehicle control method, comprising: controllingacceleration or deceleration of a vehicle on a basis of recognition ofat least one of a person who is outside the vehicle as a candidate forgetting on the vehicle or a person who is a passenger on the vehicle asa candidate for getting off the vehicle, wherein the recognition of thecandidate for getting on the vehicle is performed within a priorityrecognition area in a vicinity of a stop of the vehicle, the stop beinga scheduled location for getting on the vehicle or off the vehicle, thepriority recognition area being set on a basis of a predeterminedreference position in the vicinity of the stop.
 17. A vehicle,comprising: a controller configured to control acceleration ordeceleration of the vehicle on a basis of recognition of at least one ofa person who is outside the vehicle as a candidate for getting on thevehicle or a person who is a passenger on the vehicle as a candidate forgetting off the vehicle, wherein the recognition of the candidate forgetting on the vehicle is performed within a priority recognition areain a vicinity of a stop of the vehicle, the stop being a scheduledlocation for getting on the vehicle or off the vehicle, the priorityrecognition area being set on a basis of a predetermined referenceposition in the vicinity of the stop.
 18. The vehicle control apparatusaccording to claim 1, wherein the recognition of at least one of theperson who is outside the vehicle as the candidate for getting on thevehicle or the person who is the passenger on the vehicle as thecandidate for getting off the vehicle comprises determination of atleast one of a score indicating that the person who is outside thevehicle is the candidate for getting on the vehicle or a scoreindicating that the person who is the passenger on the vehicle is thecandidate for getting off the vehicle.